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ABSTRACT 

Determining  the  reliability  of  newly  designed  systems  is  one  of  the  most  impottant 
functions  of  the  acquisition  process  in  the  military.  Tracking  the  growth  in  reliability 
of  a  system  as  it  is  developed  and  modified  repeatedly  is  an  important  part  of  tiie  ac¬ 
quisition  process. 

This  thesis  extends  and  expands  a  reliability  growth  simulation  program  written 
previously.  It  analyzes  the  capabilities  and  limitations  of  two  discrete  reliability  growth 
models  to  determine  which  models  are  most  applicable  in  estimating  system  reliability 
under  a  variety  of  dilTercnt  growtii  patterns.  Negative  growth  patterns  are  also  consid¬ 
ered.  Tiie  result  of  this  thesis  is  a  FORTR.AN  simulation  which  enables  a  more  accurate 
estimate  of  system  reliability  using  test  data  generated  during  the  development  pliase  of 
an  acquisition  program.  '  v  -  '  '  /  '  .  r  ■  ^  ■ 
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THESIS  DISCLAIMER 


The  reader  is  cautioned  that  computer  programs  developed  in  this  research  may  not 
have  been  exercised  for  all  cases  of  interest.  While  every  elTort  has  been  made,  within  the 
time  available,  to  ensure  that  the  programs  are  free  of  computational  and  logic  errors, 
they  cannot  be  considered  validated.  .Any  application  of  these  programs  without  addi¬ 
tional  verification  is  at  the  risk  of  the  user. 
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I.  INTRODL'CTION 


The  reliability  of  a  particular  system  or  piece  ol' equipment  is  one  of  the  most  im¬ 
portant  factors  in  assessing  its  true  saiue.  A  system  with  tremendous  capabilities  tliat 
spends  an  inordinate  amount  of  lime  in  the  maintenance  facility  is  usually  less  desirable 
tlian  an  alternate  system  with  relatively  moderate  capabilities  that  has  a  much  higher 
availability.  Therefore,  accurately  measuring  the  reliability  of  a  proposed  system  during 
the  test  and  design  phases  is  an  integral  part  of  the  materiel  acquisition  process.  Tradi¬ 
tional  reliability  estimators  rely  upon  conducting  a  sullicient  number  of  tests  or  e.xpcr- 
iments  so  that  a  statistically  valid  point  estimate  or  confidence  interval  can  be 
established.  T  hese  traditional  estimates  assume  a  constant  or  lived  reliability  during  any 
particular  design  phase.  Since  reliability  will  almost  certainly  lluctunte  from  uhase  to 
piiase  of  tile  acquisition  process,  methods  are  needed  that  track  reliability  as  it  changes. 
.Additionally,  systems  being  produced  presently  are  more  complex  and  claim  a  much 
higher  reliability  than  those  produced  just  a  few  years  ago.  Verifying  high  reliai>ility  in¬ 
herently  requires  significantly  more  testing  than  does  verifying  relatively  moderate  reli¬ 
ability  since  items  are  tested  until  failure.  Therefore,  all  test  data  should  be  utili,ted  to 
tlie  utmost. 

The  process  of  testing  and  evaluating  newly  designed  pieces  of  equipment  is  very 
costly  in  terms  of  both  time  and  money.  Typically,  the  test  engineer  must  face  limitations 
in  both  of  these  resources  and  occasionally  must  compromise  on  either  the  extensiveness 
of  the  test  design  or  the  number  of  trials  to  be  run  or  both.  It  may  be  assumed  that  the 
constraints  on  time  and  money  will  not  diminish  in  the  near  future  -  if  anything,  these 
constraints  will  become  tighter.  Therefore,  methods  that  are  able  to  produce  accurate 
estimates  of  system  reliability  for  a  smaller  investment  of  resources  arc  very  desirable. 
Reliability  growth  models  arc  one  such  method. 

Reliability  growth  models  make  use  of  all  available  lest  data.  Results  from  previous 
phases  of  design  arc  combined  with  current  lest  data  so  that  a  pattern  of  reliability  is 
established.  T  hese  models  often  make  use  of  a  procedure  known  as  failure  discounting. 
Failure  discounting  involves  removing  fractions  of  pievious  failures  in  order  to  make 
allowance  for  reliability  improvement  as  the  svsi  .rn  evolves.  T  he  relialiility  pattern  es¬ 
tablished  by  a  reliability  growth  model  Iiecomes  the  b.isis  for  producing  estimates  of  the 
actual  system  reliability. 


1  here  are  two  types  of  reliability  growtli  models  --  discrete  and  continuous.  A 
continuous  model  is  based  oa  the  time  until  failure  of  the  system  under  consideration. 
Obviously,  a  number  of  distributional  assumptions  are  necessary  in  a  moilel  of  this  type. 
Discrete  reliability  growth  models  are  employed  when  test  data  leferences  the  number 
of  s\  stem  tests  and  failures  in  a  particular  design  phase.  In  order  to  use  a  discrete  reli¬ 
ability  growth  model  one  must  be  able  to  classify  a  test  result  as  either  a  laiiure  or  a 
success.  In  other  words,  the  test  data  must  be  attribute  data.  In  this  thesis,  situations 
where  test  parameters  may  classify  a  trial  as  a  partial  success  or  a  partial  failure  are  not 
allowed. 

I'his  thesis  addresses  discrete  reliability  growth  models.  The  major  objective  is  to 
analyze  the  capabilities  and  limitations  of  two  such  models.  Two  dillerent  methods  of 
discounting  previous  (ailures  arc  also  evaluated.  .A  previously  designed  simulation  pro¬ 
gram  jRef.  1:  pp.  36-Ki|  is  expanded  or  modified  as  necessary  in  order  to  more  fully  de¬ 
velop  the  two  discrete  reliability  growth  models  being  examined.  The  original  simulation 
uses  actual  reliabilities  and  Monte  Carlo  techniques  in  order  to  generate  a  random  reli¬ 
ability  growth  pattern.  The  test  data  generated  by  tiiis  growth  pattern  is  then  used  in  the 
reliability  growth  models  in  order  to  produce  estimates  of  system  reliability.  A  variety 
of  dilferent  reliability  growth  patterns  and  a  broad  spectrum  of  failure  discounting  pa¬ 
rameters  are  systematically  evaluated.  At  each  phase,  the  estimate  of  system  reliability 
is  compared  to  the  actual  reliability.  The  applicability  of  the  two  discrete  reliability 
growth  models,  to  include  limitations  and  capabilities  is  also  addressed. 

The  following  chapter  describes  each  of  the  two  discrete  reliability  growth  models 
being  considered.  .An  explanation  of  the  two  failure  discounting  methods  is  also  offered. 
•Additionally,  work  done  in  this  area  previous  to  this  analysis  is  sununarized.  Chapter  III 
discusses  dic  methodology  used  to  conduct  the  analysis,  to  include  the  computer  simu¬ 
lation.  Chapter  III  also  describes  the  various  dilferent  growth  patterns  that  were  ana¬ 
lyzed.  Chapter  IV  presents  the  results  of  the  analysis.  Chapter  V  details  conclusions  and 
recommendations. 


11.  DlSCOLiNTIiNG  PROCEDURES  AND  DISCRETE  GROWTH 

MODELS 


A.  BACKGROLfND 

In  f.his  ciuiPter,  two  discrete  reliability  growth  models  are  described.  Two  separate 
techniques  of  discounting  failures  are  also  e.xplained.  Both  methods  remove  a  fraction 
of  a  previous  failure  when  the  system  successfully  completes  a  trial  without  a  reoccur¬ 
rence  of  the  particular  failure  cause.  Reliability  is  delined  as  the  probability  of  the  sys¬ 
tem  successfully  completing  a  single  trial.  The  exact  definition  of  a  test  trial  is  much 
more  difficult  to  fix  since,  typically,  the  constitution  of  a  trial  is  the  responsibility  of  the 
testing  agency.  Normally,  a  trial  may  be  considered  as  the  exercising  of  the  particular 
system  in  a  manner  consistent  with  its  purpose.  If  a  weapons  system  is  being  evaluated, 
lor  example,  then  a  trial  could  consist  merely  of  a  single  attempt  at  target  acquisition 
or  proceed  through  target  acquisition  to  an  attempt  at  target  destruction.  The  precise 
definition  of  a  trial  depends  in  large  measure  on  the  purpose  or  goal  of  the  test.  Tor  the 
purpose  of  this  thesis  and  in  order  to  utilize  discrete  reliability  growth  models,  a  trial 
must  be  defined  so  that  it  is  discrete  and  can  be  evaluated  as  cither  a  success  or  failure. 
Partial  successes  are  not  allowed. 

/\  testing  phase  may  be  defined  as  a  number  of  trials,  one  or  more,  during  which 
the  configuration  of  the  system  is  unchanged.  Therefore,  during  a  testing  phase  the  ac¬ 
tual  system  reliability  remains  unchanged.  .A  test  phase  may  consist  of  one  or  more 
system  failures.  If  improvements  or  changes  to  the  system  are  elTected  after  each  system 
failure  (a  test-fix-test  scenario)  then  a  phase  would  consist  of  one  system  failure  and  all 
of  the  successful  trials  leading  up  to  that  failure.  Alternatively,  the  test  design  may  dic¬ 
tate  that  testing  be  continued  until  a  certain  number  of  failures  occur.  Under  this  sce¬ 
nario  (test-find-test)  the  cause  of  system  failure  is  identified  but  system  configuration 
remains  unchanged  until  the  pro-designated  lailurc  occurs.  In  general,  a  phase  will  en¬ 
compass  all  of  the  trials  between  system  configuration  changes. 

B.  FAILURE  DISCOUNTING 

Testing  conducted  during  the  initial  design  stages  of  a  particular  system  often  indi¬ 
cates  low  reliability.  Generally,  weaknesses  in  the  configuration  of  the  system  or  defects 
in  the  quality  of  its  components  cause  system  failure.  Test  designs  are  established  so 


that  the  cause  for  these  failures  can  be  identified  and  corrected.  Tlieorctically.  then,  as 
a  weakness  or  a  defect  is  identified  and.  hopefully,  corrected  the  probability  of  that 
particular  weakness  or  defect  reoccurring  should  be  reduced.  This  reduction  in  the 
probability  of  occurrence  of  a  certain  failure  cause  leads  to  improved  system  reliability. 
This  concept  is  fully  utilized  in  failure  discounting. 

In  order  to  effectively  discount  previous  failures  it  is  critical  that  the  cause  of  the 
failure  be  properly  identified.  The  level  of  detail  that  one  wishes  to  ascribe  to  this  iden¬ 
tification  process  is  dependent  upon  the  type  of  system  being  evaluated  and  the  purpose 
of  the  test.  If  a  complex  system  is  being  evaluated  then  a  failure  cause  may  be  failure 
of  a  certain  component  or  sub-component.  The  precise  element  that  caused  system 
failure  is  not  critical  but  the  ability  to  assign  a  failure  cause  to  each  system  failure  is. 
Correctly  determining  failure  cause  is  very  difllcult.  particularly  when  dealing  with  com¬ 
plex  systems.  Therefore,  it  is  conceivable  that  design  changes  do  not  improve  system 
reliability  --  in  fact,  these  changes  may  even  degrade  reliability.  To  apply  the  failure 
discounting  procedures  described  below,  one  must  be  able  to  assign  a  failure  cause  to 
every  system  failure  even  though  it  may  be  done  erroneously. 

1.  Standard  or  Straight  Percent  Failure  Discounting 

The  standard  discounting  method  will  reduce  a  previous  failure  by  a  fixed 
iunount  after  a  predesignated  number  of  trials  have  been  successfuJly  conducted  without 
a  reoccurrence  of  a  failure  due  to  the  same  cause.  This  method  has  been  detailed  fully 
in  previous  work  [Ref.  1:  pp.  14-17]  so  only  a  brief  description  and  an  example  of  its 
application  will  be  included  liere. 

To  employ  the  standard  discounting  method  one  must  specify  two  parameters 
-  the  fraction  of  a  failure  to  be  removed.  F.  and  the  discount  interval  or  the  number  of 
successful  trials,  I,  that  must  occur  before  application  of  the  discount.  For  the  purpose 
ot  this  paper  and  to  use  the  simulation  described  herein,  these  parameters  are  specified 
at  the  outset  and  remain  constant  throughout  the  test.  The  number  of  successful  tiials 
since  the  occurrence  of  the  failure  is  referred  to  as  T.  These  three  values.  F,  I.  and  T 
are  then  used  to  discount  previous  failures  at  the  end  of  eacli  phase. 

■Mathematically,  the  standard  discounting  method  is  given  by  liquation  2.1; 

DISCOUSTED  FAILURE={[  -  (2.1) 

The  term  1\T  in  the  above  equation  refers  to  the  integer  portion  of  the  arszument 

r 

( ~)  •  As  such,  it  the  number  of  successful  trials,  since  the  failure  cause  last  occurred. 


1.  IS  less  than  the  number  of  trials  required  by  the  discount  interval.  I.  then  /.\’/(  —  ) 
will  be  zero  thus  causing  no  discounting  to  bo  applied.  .Additionally,  should  a  lailure 
cause  reoccur  then  the  value  of  T  is  returned  to  zero  thus  removing  any  previous  dis¬ 
counting  that  may  have  been  applied.  This  ability,  in  elfect.  acts  as  a  penalty  factor. 
If  a  failure  cause  should  reoccur  then  previous  design  changes  may  not  have  been  ef¬ 
fective  and  thus  any  prior  failure  discounting  could  be  unwarranted,  fhe  rate  and 
amount  of  failure  discounting  can  be  controlled  by  altering  F  and  I.  If  the  system  under 
evaluation  is  relatively  simple  and  a  proven  failure  identilication  process  is  in  place,  then 
a  large  discount  to  system  failures  may  be  warranted.  This  will  result  in  a  prediction  of 
rapid  reliability  growth. 


Table  1.  STA.NDARD  DLSCOL  NT  METHOD  EXAMPI  E 


.\  small  example  of  the  application  of  the  standard  discount  method  will  aid  in 
clarifying  its  abilities.  Consider  the  data  in  Table  1.  This  test  data  is  purely  fictitious 
and  is  intended  solely  to  illustrate  the  use  of  the  standard  discount  method.  We  will 
assume  that  improvements  or  repairs  are  elfccted  after  each  failure  so  actual  system  re¬ 
liability  is  not  constant. 

Table  I  represents  three  phases  of  a  test-li.x-tcst  scenario  where  two  failure 
causes,  X  and  Y,  have  been  identified.  It  should  be  noted  that  under  a  test-lix-test  plan. 


a  phase  will  tcrnhnate  with  system  failure.  The  result  ol'  each  trial  is  listed  as  cither  a 
success  or  failure  for  a  particular  failure  cause. 

The  two  parameters  necessary  to  employ  the  standard  discount  method  are  ih.e 
Iraction  of  failure  to  he  removed.  F.  and  the  discount  interval,  I.  Assume  that  F  =  o.Ju 
and  I  =  3.  System  failure  number  two.  attributed  to  failure  cause  'i'.  terminates  phase- 
two.  .‘\t  that  point  the  failure  due  to  cause  that  terminated  the  fust  pliase  has  had 
three  successive  successful  trials  so  failure  discounting  can  be  applied: 

.  1 D  JUST  ED  FAILURE  =  ( 1  -  0.50)‘  i '  =  0.50 

Thus,  for  reliability  computations  at  the  conclusion  of  the  second  phase,  the  failure  that 
occurred  during  phase  1  is  only  counted  as  one-hall' of  a  lailure. 

Phase  tiiree  tcrimnates  with  a  failure  due  to  cause  X.  The  value  Ibr  T  then  is 
set  to  zero  and  tiius  when  applied  to  the  failure  that  occurred  in  phase  one.  lull  value  is 
restored.  At  the  end  of  phase  two  this  failure  had  been  discounted  to  0.50  but.  because 
the  failure  cause  reoccurred,  this  discount  is  viewed  as  unwarranted  and  the  lailure  is 
returned  to  one.  However,  at  the  end  of  phase  three  failure  cause  'i'  has  had  seven  suc¬ 
cessful  trials.  Therefore,  discounting  is  applied: 

.  I DJUSTED  FAILURE  =  ( 1  -  0.50)''''^ T ’  =  (0.50)"  =  0.25 

I  hereforc.  when  computing  reliability  estimates  at  the  end  of  phase  three,  only  2.25 
''Sstem  failures  (failure  number  one  plus  failure  number  three  plus  the  discounted  value 
of  failure  number  two)  are  considered  instead  of  the  three  failures  that  actually  occurred. 

1  he  standard  discoun’’  method  described  is  very  lle.xible  because  of  its  two  pa¬ 
rameters.  Setting  values  for  F  and  I  does,  however,  require  a  good  deal  of  professional 
judgement  and  e.xpertise.  To  avoid  conllicts  in  setting  such  values  David  K.  I.loyd  pro¬ 
posed  an  alternate  method  of  discounting  failures  [Ref.  2).  Ihc  Lloyd  method  is  de- 
'.^iibed  in  the  next  section. 

2,  LI(i\d  Failure  Di.sconnting 

Discounting  previous  failures  using  the  I.loyd  discounting  method  is  based  on 
the  premise  that  there  should  be  some  sort  of  statistical  basis  fnr  de'  -rmining  how  much 
a  precious  failure  slioiild  be  discounted.  Lloyd  oilers  the  upper  conlioence  limit  of  the 
probability  that  a  failure  cause  will  occur  agtiin  as  the  discounted  value  of  its  rcspectic  e 
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failure.  The  confidence  limits  for  this  proliahility  are  bused  on  the  mimbcr  of  successful 
trials  since  the  failure  cause  last  occurred.  L'nder  a  te.st-iix-test  scenario  this  is  ecjui\alent 
to  the  number  of  successful  trials  since  the  last  failure.  Tliis  number  was  delined  in  the 
description  of  the  standard  discount  method  as  f.  In  order  to  implement  Lloyd  failuic 
discounting,  one  must  set  the  level  of  confidence  desired  for  the  confidence  bound.  .Since 
the  method  is  applied  on  every  trial  there  is  no  requirement  to  specify  an  inters  ai.  al¬ 
though  as  will  be  explained  in  a  later  chapter,  specifying  an  interval  for  the  Lloyd 
method  can  be  done  and  may  be  advantageous  in  some  situations. 

The  Lloyd  discount  method  may  be  mathematically  expressed  as  follows: 


ADJUSTED  FAIU'RE^  1  -(1  -  C/)r  iJ'T>0 


(2.1) 


In  tlie  above  equation  Cl  is  delined  as  the  level  of  confidence  desired  .and  T  is  tire 
number  ot' successful  trials.  If  T  has  a  value  of  zero  then  the  value  of  the  adjusted  failure 
is  set  to  one.  Setting  the  value  of  the  adjusted  failure  equal  to  one  when  T  equals  zero 
gives  the  Lloyd  method  the  ability  to  restore  ail  of  the  value  of  a  discounted  failure 
should  its  failure  cause  reoccur. 


Table  2.  LLOYD  DISCOUNT  METHOD  EXAMPLE 
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Table  2  illustrates  the  use  of  the  Lloyh  method.  The  test  data  is  the  same  data 
used  to  demonstrate  the  standard  diseount  method.  Since  the  Lloyd  method  i‘>  applied 
on  every  trial  a  I'onrth  cnlunm  has  been  added  to  the  ttdsle  retlecting  the  current  value 
of  the  adjusted  lailure.  Let  us  assume  that  the  confidence  interval  desiied  in  this  case 
'vvas  .90. 

If  reliability  is  computed  at  the  completion  of  the  third  phase  then  2.2S  system 
lailures  would  be  used  under  the  Lloyd  discounting  method  as  opposed  to  the  actual 
value  of  three  that  occurred.  The  last  colunm  of  the  table  indicates  the  speed  that  failures 
become  discounted  with  the  Lloyd  method. 

I'he  two  discrete  reliability  growth  models  that  will  be  e.xamined  are  described 
in  the  ne.\t  section.  IZach  model  makes  use  of  discounted  lailure  data  and  each  has 
demonstrated,  through  simulation,  the  ability  to  estimate  constant  and  concavely  in¬ 
creasing  reliability  with  reasonable  accuracy. 

C.  DISCRETE  RELIADILITY  GROWTH  MODELS 

Both  cl'  the  discrete  reliability  growth  models  to  be  presented  assume  a  constant 
system  reliability  within  a  phase.  The  first  model  to  be  e.xamined  will  be  tlie  .Maximum 
Likelihood  Estimate  with  Failure  Discounting  (MLEFD).  This  model  is  merely  a  deriv¬ 
ative  of  the  conventional  single  phase  maximum  likelihood  estimate.  The  single  phase 
ma.vimum  likelihood  estimate  considers  only  test  data  generated  during  llte  current 
phase.  The  .MLEFD.  conversely,  is  cumulative  in  that  it  considers  all  of  the  available 
test  data.  The  test  data  from  eaily  phases  during  which  the  actual  system  reliability  was 
dilferent  requires  adjustment.  I'hus,  the  use  ol' failure  discounting  techniques  provides 
a  method  by  which  failures  occurring  early  in  the  development  process  can  lie  assim¬ 
ilated  into  current  reliability  estimates.  The  second  model  is  a  regression  model  based 
on  an  exponential  single  phase  reliability  estimate.  It  estimates  reliability  for  each  phase 
using  an  exponential  model  and  then  performs  a  linear  regression  on  these  estimates:  to 
obtain  a  current  estimate.  The  simulation  program  that  was  written  to  evaluate  these 
two  models  was  designed  so  that  failure  di.scounting  may  or  may  not  be  invoked  with 
either  model. 

Before  olfering  a  general  description  of  the  two  models  it  is  impoitant  at  this 
juncture  to  digress  for  a  moment  and  discuss  the  e.xact  metliodology  that  will  lie  used  to 
incorporate  discounted  failures  Into  the  various  reliability  estimates.  As  was  seen  in  the 
section  prior,  discounted  or  adjusted  failures  are  rarely  if  ever  integer-valued.  While  this 
does  not  present  a  problem  for  the  .MLEFD  reliability  growth  model  it  does  pose  an 


obstacle  to  using  the  exponential  regression  model.  The  convention  devised  in  Captain 
Drake  s  thesis  [R.et'.  1:  p.  33|  will  be  continued  in  this  paper.  The  number  of  trials  up  to 
and  including  a  particular  failure  will  be  divided  by  the  adjusted  failure  so  tliat  an  ad¬ 
justed  number  of  trials  is  computed.  The  adjusted  failure  value  is  then  returned  to  one 
I'rom  its  previous  fractional  value.  The  formula  for  computing  adjusted  trials  is  then: 


ADJUSTED  TRLILS 


TOTAL  I'RIAl.S _ 

ADJUSTED  FAILURE 


(2. ') 


fhe  appeal  of  this  method  is  that  the  result  is  an  integer  number  of  tailures  while  main¬ 
taining  a  constant  ratio  of  nuntber  of  failures  to  number  of  trials.  While  the  adjusted 
number  of  trials  may  not  be  integer-valued  they  may  be  adjusted  to  integers,  if  neccs'^ars , 
through  rounding  with  a  much  smaller  elTcct  on  reliability  computations  than  lonnding 
adjusted  failures. 

If  a  failure  had  been  discounted  (using  either  discounting  methudi  iiom  one  to  .2:'. 
for  example,  and  the  number  of  successful  trials  since  the  failure  occtnred  was  ten  then 
the  ratio  of  failures  to  successes  would  be  .25  to  ten.  Using  the  method  described  above, 
the  adjusted  failure  and  actual  trials  are  divided  by  the  avljusied  failure  yielding  a  failure 
to  success  ratio  of  one  to  forty.  Thus,  the  number  of  trials  has  been  adjusted  from  ten 
to  forty  and  the  number  of  failures  has  been  returned  to  one  from  .25.  Since  the  ratio 
of  failures  to  successes  remains  the  same  reliability  cornputatioms  will  not  be  alfected. 
Both  of  the  models  described  below  will  make  use  of  this  method  in  computing  reliability 
estimates. 

1.  Ma.ximuiit  Likelihuud  Estimate  with  Failure  Discounting 

The  traditional  estimate  of  system  reliability  is  the  maximum  likelihood  esti¬ 
mate.  Letting  R  denote  the  estimate  of  reliability,  the  maximum  likelihood  estimate  may 
be  expressed  as  follows: 

n  =  total  TRI  \LS  -  T'^  'Al.  EAIH  RES 
TOTAL  TRIALS 

or  alternatively: 

n  ^  SUCCESSFUL  TRIALS 
TOTAL  TRIALS 


As  stated  previously,  the  maximum  likelihood  estimate  assumes  a  constant  re¬ 
liability  between  phases.  It  also  requires  a  large  number  of  trials  to  accurately  estimate 


tlie  urulcrlyme  system  reliability.  Failure  (Jiseouiitiiig  is  used  to  make  previous  test  data 
compatible  with  current  results  and  thus  increase  the  total  number  of  trials  available. 
However,  since  the  actual  system  reliability  is  unknown  it  becomes  a  very  nen-trivial 
task  to  select  the  appropriate  discounting  method  and  the  correct  p.arameters. 

Vv'hen  presious  test  data  is  made  available  through  the  use  of  a  discounting 
routine  the  equation  for  estimating  the  system  reliability  becomes; 

,•  ^  TOTAL  TRIALS  -  TOTAL  ADJUSTED  F.ULL  RKS 

TOTAL  TRIALS 

or  if  the  convention  of  restoring  tlie  number  of  failures  to  integer  values  is  adopted 

,,  ^  TOTAL  ADJUSTED  TRIALS  -  TOTAL  LAILCRES  p  , 

TOTAL  ADJUSTED  TRIALS 

To  utilize  the  MLFFD  reliability  growth  model  one  first  records  ail  of  the  test 
data  from  the  cuirent  phase.  The  discounting  method  to  be  used  is  then  perl'ormed  on 
tite  previous  test  data  and  an  adjusted  failure  value  is  computed  for  each  previous  I'ailurc. 
The  adjusted  number  of  trials  for  each  failure  is  then  computed  using  Fquation  2.3.  T  he 
estimate  of  reliability  for  the  current  phase  is  then  calculated  using  Equation  2.4  above. 

One  deficiency  ol'  the  maximum  likelihood  estimate  is  that,  when  testing  is  ter¬ 
minated  after  a  Fixed  number  of  failures,  the  .\ILE  is  inherently  a  biased  estimator  of  the 
actual  reliability.  The  expected  value  of  the  maximum  likelihood  estimate  has  been  de¬ 
rived  in  previous  work  [Ref.  3:  p.34|.  It  may  be  expressed  mathematically  as  follows: 

=  I  +  (-|-)ln(l  - /^)  -  lull-/?)  (2.3) 

where  R  represents  the  actual  reliability  of  the  system.  Ihe  bias.  D(R},  in  the  estimate 
may  be  calculated  using  the  formula; 

mil)  =  EIR]-R 

■fable  3  below  depicts  the  performance  of  the  maximum  likelihood  estimate  lor 
dilferent  values  ol' actual  reliability.  R.  I  he  bias  is  inherent  to  the  estimate  in  that  it  will 
be  present  regardless  of  the  vtilidity  of  the  assumption  of  constant  phase  relial.iility. 

file  use  of  failure  discounting  can  olFset  the  elfect  of  bias  as  well  as  handie 
changing  system  reliability.  However,  this  [luts  a  lot  of  additional  weight  on  the  ability 
to  correctly  select  the  best  method  and  choose  the  proper  parameters.  Alternatively,  if 
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an  estimator  of  system  reliability  had  to  be  biased  one  would  want  it  to  be  conservatively 
biased  as  is  the  MLL'.  L'nderestimating  actual  system  reliability  is  tiormally  less  costlv 
than  overestimating.  However,  a  less  biased  estimate  would  be  benellcial. 

Table  3.  INHERENT  BIAS  IN  THE  MAXIMUM  LIKELIHOOD  ES  1  LMATE 


RELIABILITY 

BiR) 

O.307 

-0.193 

0,60 

0.389 

-0.21  1 

0.7(.) 

0.4S4 

-0.216 

o.SO 

0..598 

-0.202 

0.00 

0.744 

-0. 156 

0.95 

0.842 

-(».10S 

('.892 

-o.oTS 

0.99 

i  0.953 

-o.o3' 

2.  E.xponeiitiai  Regression  Estimate 

The  e.xponential  regression  reliability  growth  model  was  developed  by  H. 
ChernolT  and  W.M.  Woods  and  based  on  the  exponential  single  phase  estimate  [Ref. 
-IJ.  The  derivation  of  this  model  has  been  developed  fully  in  previous  work.  [Ref.  5;  pp. 
3-.5|  so  for  the  purposes  of  this  paper  a  general  sununary  of  results  will  be  olfered. 

The  basic  form  of  the  exponential  single  phase  estimate  for  a  phase  k  i.c..  after 
the  kth  improvement  or  alteration  has  been  made,  is  stated  in  Equation  2.6; 

R,  =  (2.6) 

where  is  the  reliability  estimate  and  the  coelTicient  of  the  exponential  term.  'i'.  can 
be  calculated  as  follows; 

y,  =  l+T  +  y+...+  ./.v,>2 

where  A'*  is  the  number  of  trials  up  to  and  including  the  first  lailure  in  phase  k.  If  .V. 
equals  one  then  is  defined  equal  to  zero.  It  should  be  noted  that  this  estimate  was 
developed  for  the  special  case  of  testing  until  the  first  failure  is  encountered.  If  system 
failure  occurs  on  the  first  trial  (A*  =  I)  then  )\  will  equal  zero  and  system  reliability  will 


be  cstiinatcJ  as  1  -  a"  =  1-1  =  <>.  IT  failure  occurs  on  t!;c  ilnrJ  tnai,  then  hic  c\[io- 
neiuial  single  phase  estiinate  wouiJ  yield  I  —  <r  ''T’  = 

In  order  to  evaluate  the  properties  of  the  evponcnTial  single  ph.ase  estmiate  a 
small  simulation  was  designed.  Geometric  test  data  witii  actual  reliabilitv  R  were  gener¬ 
ated  using  a  transformation  of  a  standard  L  nii'orm(().  I )  random  number.  Ten  thousand 
sets  of  geometric  data  were  generated  for  each  value  of  R.  fable  4  represents  the  results 
of  this  simulation.  .Also  included  in  the  table  is  a  simulation  of  the  ma.ximum  likelihood 
estimate.  l  /G,  using  the  same  test  data. 

The  values  generated  for  the  mean  of  tlie  maximum  likelihood  estimate.  R  . 
conform  I'airly  well  with  the  theoretical  results,  thus  tlie  simulation  itself  is  accurate.  The 
values  generated  for  tlie  exponential  estimate,  R  .  while  still  underestimating  the  actual 
reliability,  are  clearly  less  conservatively  biased  than  the  maximum  likelihood  estim.ites. 
file  sample  variance  and  mean  s^piare  error  of  each  estimate  is  also  represented  m  tlie 
table.  Tile  mean  scjuare  error  of  each  estimate  is  the  sum  of  the  variance  and  tlie  square 
of  tlie  bias.  It  can  be  seen  that  as  the  actual  reliability  exceeded  .70  the  mean  square 
error  for  the  exponential  estimate  became  less  than  the  mean  square  error  of  the  maxi¬ 
mum  likelihood  estimate.  Therefore,  the  advantage  of  a  lower  variance  for  the  maximum 
likelihood  estimate  at  values  of  actual  reliabilitv  below  .90  is  olfset  bv  the  greater  bias. 


Maximum  Likelihood  Lstimate 


n.3n7 


()..TSh 


0.484 


n..^o,s 


(I, O') 


0.842 


0.802 


( 1. 054 


O.IOl 

0.141 

0.1 1  1 

0,157 

0.1 15 

0.163 

0.  107 

0.140 

O.O70 

0. 103 

o.o.so  i 

0.0.34 

0.040 

0.014 

0,015 

R 


0.357 


0.451 


0.547 


O.'K'  I 


Table  4.  PROPERTIES  OF  THE  EXPONENTIAL  RELIACILII Y  ESTIMATE 


Exponential  Lstimate 


R  r,,r(R)  MSBR)  R 


0.302 


0.387 


0.480 


0.595 


0.15^ 


I '.  I  t'b 


0. 1 40 


0.I2I 


0. 14(1 


0.881 


0.021 


o.on.s 


0.047 


0031 


o.o'3 


o.oi : 


The  cxponciuial  regression  reliability  growth  model  developed  by  Chcrnoli'and 
\\'ooJs  uses  the  teehmque  of  linear  regression  to  estimate  the  eoellkient  of  the  expo¬ 
nential  term.  The  model  can  be  expressed  mathematically  as  Ibilows; 

il:,  =  1  12.7) 

In  the  equation  above  k  denotes  the  testing  phase  being  used  to  compute  tlie 
system  reliability,  a  and  p  are  computed  at  the  end  of  each  phase  using  the  techniques 
of  linear  regression  thus  yielding  an  estimate  of  system  reliability.  The  exact  derivation 
of  the  linear  regression  formula  for  at  and  p  is  detailed  in  other  sources  [Ref.  3:  p.  2]  and 
is  provided  at  .Appendix  A  for  the  interested  reader. 

This  model  requires  that  both  failures  and  trials  be  integers.  I  herefoic,  when 
adjusted  trials  are  computed  using  Equation  2.3,  the  value  obtained  must  be  rounded  to 
the  nearest  mtegor.  Since  linear  regression  is  employed,  this  model  has  the  abilitv  to 
track  clianging  reliability  without  the  use  of  lailure  discounting. 

D.  SUMMARY  OF  PREVIOUS  WORK 

.Analysis  of  reliability  growth  models  has  been  a  continuing  process,  and  many 
advances  have  been  made  in  deterrning  their  properties  [Refs.  6. 7].  It  would  require  many 
pages  to  summarize  all  of  the  previous  woik  in  this  area,  for  tlte  purposes  of  this  paper, 
the  summary  will  detail  only  those  results  which  have  a  direct  bearing  on  the  simulation 
being  used  to  evaluate  the  two  reliability  growth  models. 

.A  simulation  to  evaluate  these  models  was  constructed  by  Captain  James  Drake 
[Rel.  1:  pp.  3o-40|.  This  simulation  was  written  to  lumdle  growth  patterns  where  actual 
reliability  was  constant  or  growing  at  a  lixed  rate.  Since  one  of  the  objectives  of  this 
ptij-ier  is  to  expand  the  number  of  reliability  growth  patterns  that  can  be  sintulated  it  is 
important,  at  this  time,  to  summarize  Captain  Drake's  work  to  include  the  assumptions 
neccssarv  to  run  the  simulation  and  some  of  the  limitations. 

Many  reliability  estimators  are  too  mathematically  complex  to  allow  closed  form 
solutions  of  their  properties.  In  order  to  compare  these  properties  computer  simulation 
is  one  of  the  only  techniques  available.  1  he  simulation  constructed  by  Captain  Drake 
builds  a  reliability  growth  pattern  and  generates  test  data  based  on  known  system  reli¬ 
abilities  using  the  geometric  distribution.  Ihc  discrete  reliability  growth  model  being 
evaluated  has  access  to  this  test  data  for  reliability  estimation,  fhc  test  data  generated 
by  the  computer  simulation  is  typically  the  number  of  trials  up  to  and  including  lailure 
and  the  cause  ol  system  failure  for  each  predcsignated  failure  in  a  phase. 


The  inputs  required  to  run  the  original  simulation  are  (Ref.  1:  pp.  40— ll|: 

1.  Number  of  testing  phases. 

2.  Number  of  system  failures  allowed  in  each  phase. 

3.  Number  of  possible  failure  causes. 

4.  Probability  of  non-occurrence  of  a  particular  failure  cause  in  the  first  phase. 

5.  Reliability  growth  fraction. 

6.  Discounting  option  and  parameters. 

Several  assumptions  were  necessary  to  produce  an  adeijuate  simulation  based  on 
these  inputs.  These  assumptions  are  grouped  into  two  categories  -  reliability  growtli 
pattern  assumptions  and  failure  cause  assumptions.  The  assumptions  required  for  simu¬ 
lating  the  rci;ability  growth  patter’'  are  [Ref.  1:  pp.  37-38]: 

1.  The  relitibility  growth  pattern  is  non-decreasing. 

2.  Systent  reliability  changes  only  at  phase  boundaries. 

3.  Equipment  improvements  are  implemented  inunediately  after  a  phase  ends  and 
before  any  further  testing. 

4.  Failure  causes  are  corrected  only  at  the  end  of  a  phase. 

5.  Each  design  improvement  or  repair  removes  a  fi.xed  fraction  of  the  probability  of 
reoccurrence  for  the  corresponding  failure  cause. 

These  assumptions  narrowed  the  scope  of  the  reliability  growth  patterns  that  could 
be  evaluated.  The  first  assumption  does  not  allow  the  model  to  track  any  reliability 
growth  patterns  that  e.xhibit  periods  of  declining  reliability.  While  for  the  most  part  ac¬ 
tual  system  reliabilities  will  be  non-decreasing,  this  restriction  does  preclude  a  complete 
evaluation  of  the  reliability  estimators  under  consideration.  Assumptions  two  and  three 
are  consistent  with  the  previously  stated  definition  of  a  testing  phase.  .Assumption  three 
does  not  allow  modeling  of  long  term  design  changes  that  while  indicated,  may  be  post¬ 
poned  for  whatever  reason.  Assumption  five  represents  the  method  that  is  used  in  the 
simulation  to  model  the  impact  ol' design  changes  or  improsements.  .According  to  this 
assumption,  any  improvement  or  design  cljange  will  remove  a  fixed  fraction  (user  input 
number  five)  of  the  probability  of  occurrence  for  a  failure  cause.  The  underlying  as¬ 
sumption  is  that  all  improvements  arc  equally  ellective. 

These  assumptions  were  necessary  to  generate  the  type  of  reliability  growth  patterns 
desired  in  Captain  Drake  s  thesis.  I3y  introducing  the  concept  of  fixed  phase  reliability 
and  modifying  the  original  simulation  accordingly,  four  of  these  five  assumptions  were 


no  longer  necessary.  The  e.xaet  methodology  used  to  etiect  this  niodilieation  will  he  de¬ 
tailed  in  the  next  chapter. 

The  last  category  of  assumptions  in  the  original  simulation  are  the  failure  cause 
determination  assumptions.  These  assumptions  are  jRe!'.  I  :  p.  ,'d|; 

1.  There  exists  a  finite  number  of  possible  failure  causes. 

2.  Tach  failure  cause  has  a  fixed  probability  of  occurrence  in  each  phase. 

3.  System  reliability  can  be  modeled  as  a  series  system  of  the  failure  causes. 

4.  Each  failure  cause  is  stochastically  independent  of  the  other  failure  causes. 

.Assumption  three  dictates  the  method  by  which  the  actual  system  reliability  is 
computed  in  the  simulation.  Since  the  occurrence  ol'a  failure  cause,  by  definition,  means 
system  failure,  modeling  the  system  as  a  series  is  acceptable.  This  implies  then  that  the 
actual  system  reliability  may  be  mathematically  expressed  as: 
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In  the  above  equation  P,  is  the  probability  of  occurrence  of  the  ith  failure  cause  and 
n  is  the  total  number  of  failure  cau.ses. 

The  simulation  is  able  to  generate  descriptive  statistics  about  each  of  the  reliability 
growth  models  being  examined.  These  statistics  include  the  mean,  its  confidence  interval 
and  the  standard  deviation  for  each  reliability  estimate  at  each  phase.  In  order  to 
produce  these  statistics  the  simulation  should  be  replicated  a  sudicient  number  of  times. 
The  mean  and  the  standard  deviation  are  computed  in  the  usual  manner.  I'he  confidence 
interval  calculation  assumes  a  normal  distribution.  Tor  the  purposes  of  this  paper.  300 
replications  of  each  set  of  input  data  were  conducted  so  the  normality  assumption 
should  be  reasonable. 

(Jne  of  the  limitations  of  the  model  in  addition  to  the  restricted  number  of  reliability 
growth  patterns  that  could  be  analyzed  was  in  the  application  of  the  discounting  meth¬ 
ods.  The  lull  range  of  input  parameters  could  not  be  explored  due.  in  large  measure,  to 
computer  limitations.  This  limitation  typically  manifested  itself  at  relatively  high  system 
reliability.  If  a  failure  cause  with  a  very  low  probability  of  occurrence  caused  system 
failure  early  and  if  the  input  parameters  for  the  discounting  method  were  set  such  tliat 
a  rclativelv  larce  fraction  of  this  failure  was  discounted  over  a  relativelv  short  number 


of  successful  trials  then  it  was  possible  for  this  early  failure  to  be  discounted  to  essen¬ 
tially  zero.  When  the  adjusted  trials  were  calculated  utilizing  liquation  2.3 

(ADJUSTED  TRIALS  =  .  *  )  •  then  the  value  approached  infin- 
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ity  creating  a  system  error  in  the  computer. 

This  problem  was  solved  by  specifying  a  lower  bound  for  the  adjusted  I'ailure.  If  the 
failure  was  discounted  so  that  its  adjusted  value  was  less  than  .0000f)01  then  the  adjusted 
lailure  was  set  to  .000<J()01.  This  rnodilication  allowed  the  full  range  of  input  parameters 
for  the  discounting  methods  to  be  evaluated.  The  eHcct  on  the  estimate  of  system  reli¬ 
ability  is  negligible. 


HI.  METHODOLOGY 


One  of  the  major  objectives  of  this  paper  is  to  expand  the  simulation  described  in 
the  previous  chapter  so  chat  a  greater  variety  of  reliability  growth  patterns  could  be  used 
to  evaluate  the  two  reliability  growth  models.  This  was  accomplished  through  a  modifi¬ 
cation  of  the  simulation  that  allowed  the  user  to  input  the  desired  .system  reliabilities  at 
each  phase.  This  chapter  will  detail  the  method  that  was  used  to  elfect  this  modification 
along  with  a  description  of  the  reliability  growth  patterns  and  failure  discounting  options 
that  were  used. 

Fl.XED  PHASE  RELIABILITY 

One  of  the  limitations  of  the  original  simulation  was  that  only  constant  reliafiility 
or  reliability  that  increased  at  a  constant  rate  could  be  modeled.  If  a  predesignated  fail¬ 
ure  cause  resulted  in  a  failure  then  that  failure  cause  s  probability  of  occuircnce  on  any 
succeeding  phase  was  reduced  by  a  predetermined  constant  amount.  Thus,  once  the  user 
of  the  simulation  established  the  initial  inputs  for  phase  one,  he  had  very  limited  control 
over  the  actual  system  reliabilities  in  the  subseiiuent  phases. 

In  order  to  evaluate  the  two  reliability  growth  models  under  a  greater  variety  of 
potential  reliability  growih  patterns  the  concept  orfi.xed  piiase  reliability  was  introduced 
as  an  option  to  the  original  simulation.  With  this  modil'ication  of  the  program,  the  user 
has  the  ability  to  fi.x  the  actual  system  reliability  at  each  phase.  Thus,  the  relialrility 
growth  models  can  be  used  to  estimate  system  reliability  under  any  reliability  growth 
pattern  the  user  desires. 

Incorporating  li.xed  phase  reliability  into  the  simulation  necessitated  an  expansion 
of  the  user-provided  input.  The  basic  premise  used  in  computing  actual  system  reliability 
remains  the  same.  This  premise  is  that  system  reliability  is  the  product  ol'  the  probabili¬ 
ties  of  success  of  the  failure  causes  (  Equation  2.8,  R  =  fjll  ~  )•  Originally,  the 

simulation  required  that  the  user  input  the  probability  of  success  for  each  hiilure  cause 
for  the  iirst  phase.  After  that,  the  liactional  improvement  factor  dictates  the  actual  s\s- 
tem  reliability  in  succeeding  phases.  If  the  option  of  fixing  the  phase  reliabilities  is  cho¬ 
sen.  then  the  user  must  specify  the  probability  of  success  for  each  failure  cause  in  each 
phase.  Equation  2.8  is  then  applied  at  each  phase  to  \ield  the  system  reliability. 


riie  table  below  illustrates  the  inputs  that  would  be  required  to  produce  a  growth 
pattern  that  initially  declines  and  then  increases  rapidly.  The  scenario  J.:pictod  is  a  sim¬ 
ple  one  involving  live  test  phases  with  one  lailure  per  phase  and  two  failure  causes,  X 
and  h'.  Desired  reliability  is  user  dciernrined. 

Table  5.  FIXED  PHASE  RELIABILITY  EXAMPLE 


DESIRED 

USER  INPL  IS 

ACT!  .\L 

PHASE 

RELIABIL- 

PROBABILITY  OE  SLCCESS 

RE!. I- 

ITY 

CAL  SE  X 

CAl  SE  Y 

.ABi  i.n  h 

1 

O.Sl.l 

0.90 

0.89 

0.80  1 

2 

l).70 

0.83 

0.84 

0.09' 

3 

0.77 

0.84 

0.04' 

■* 

0.0 

0.S4 

0.,89 

O  '.qs; 

O.Vl  I 

0.94 

o.9() 

o.nip 

The  actual  reliability  for  each  phase  was  computed  using  Equation  2.8.  Consider 
phase  one.  for  example.  The  desired  system  reliability  is  U.SO.  To  achieve  this  reliability, 
the  probabilities  of  success  for  failure  causes  X  and  Y  were  input  as  U.9U  and  0.89  re¬ 
spectively.  1  his  yielded  a  computed  system  reliability  of: 


=  Hd  -  =  -90  X  .89  *  0.801 


Figure  1  is  a  graphical  representation  of  the  reliabilities  computed  in  the  table.  The 
original  simulation  could  not  have  produced  this  reliability  pattern  since  declining  reli¬ 
ability  is  present.  I  his  scenario  may  occur  for  systems  during  a  portion  of  their  devel¬ 
opment,  While  it  is  true  that,  in  the  long  run,  most  systems  exhibit  non-decreasing 
reliabilities  it  is  important  to  consider  decreasing  reliability  when  evaluating  giowth 
models. 

In  order  to  invoke  this  ability  to  specify  reliability  at  each  phase,  the  number  ol  user 
provided  inputs  increased  from  two  to  ten.  If  there  had  been  two  failures  allowed  in  each 
of  the  live  testing  phases  then  the  user  inputs  would  have  increased  IVom  ten  to  twenty. 
I  herefore.  while  the  option  of  allowing  the  user  of  the  simulation  to  control  the  actual 
system  reliability  at  each  phase  significantly  increases  the  variety  of  diflerent  reliability 


growth  patterns  that  can  be  analyzed,  a  rather  large  increase  in  llie  volume  of  input  data 
is  required. 

B.  RELIABILITY  GROWTH  PATTERNS 

To  more  fully  evaluate  the  properties  and  trends  of  the  two  reliability  growth 
models  described  in  the  previous  chapter,  it  was  necessary  to  analyze  their  performance 
with  respect  to  a  variety  of  dilVerent  reliability  patterns.  T  he  eight  growtli  patterns  that 
wore  analyzed  in  this  paper  will  be  presented  in  this  section. 

In  conjunction  with  the  reliability  growth  patterns,  another  important  impact  on 
the  performance  of  the  reliability  estimators  was  the  choice  of  discounting  method.  The 
table  below  depicts  the  various  combinations  of  parameters  for  the  standard  discount 
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method  that  were  analyzed.  The  discount  fraction  is  the  amount  of  a  previous  failuie 
that  is  removed  eacit  time  the  discount  method  is  applied  wliile  the  discount  intei\al  is 
the  number  of  successful  trials  between  a['plications.  The  Lloyd  discount  method  onis 
has  one  parameter,  tlie  conlidence  interval.  Tor  the  purposes  of  this  paper,  the  L!o>d 
method  was  applied  with  conlidence  values  of  .S  and  .9. 


Table  fi.  STA.NDARD  DlSCOl'NT  METHOD  C  OMBINATIONS 


Sr.AND.ARD  DISCfOTM  METHOD 


DISCOUNT  FR^ACTION  DISCOL  NT  IN  fERV.X  L 


I  he  sceranos  used  to  analyze  the  reliability  growth  models  in  this  paper  involved 
ten  testing  phases,  i.e..  testing  v.as  conducted  until  ten  changes  had  been  nude.  .A  total 
of  five  failure  causes  were  considered  in  each  test  and  a  limit  of  one  failure  per  phase 
was  established.  The  simulation  is  able  to  handle  any  number  of  test  phases,  any  num¬ 
ber  of  lailure  causes,  and  any  number  of  failures  per  phase.  The  only  lintilation  is  the 
capacity  of  the  computer  on  which  the  simulation  is  run.  .Another  practical  limitation 
involving  the  fixed  reliability  option  is  tiie  volume  o(  input  data  that  would  accoinpanv 
a  test  design  involving  numerous  phases  or  failure  causes.  .\s  can  be  seen  by  the  reli¬ 
ability  growth  patterns  and  user  input  tables  following,  the  probabilities  of  success  for 
the  failure  causes  were  allowed  to  fluctuate.  The  only  requirement  was  that  the  reliability 
growth  pattern  established  accurately  rellected  the  desired  growth  pattern. 

Figure  2  and  1  able  7  describe  the  first  reliability  growth  pattern.  This  is  not  a  very 
conventional  pattern  in  that  most  estimators  assume  concave  sjrowth  patterns  of  the 


ibrni  ''hown  in  fieures  5  nrui  n.  However,  ihi,';  t>pc  of  reliability  growth  pattern  weniJ 
no":  be  unusual  in  situations  where  the  exact  nietlioJ  or  technology  required  to  ...'ircet 
a  huiure-causing  dei'cct  is  not  iiuinediately  a\ailable  but,  as  the  system  evohes  and  per¬ 
sonnel  Iseconie  more  lanuliar  with  it.  the  the  lailure  correction  process  proceeds  more 
eli'icientiy. 

I’igure  3  depicts  a  decreasing  reliability  growth  pattern.  In  this  instance,  an  at¬ 
tempted  improvement  actually  caused  system  reliability  to  decrease.  I  his  situation, 
which  IS  not  eUcctively  addressed  by  any  conventional  reliability  estimators,  could  pos- 
sibiy  occur  in  experimental  or  technologically  adwinced  svstems  uhere  the  complete 
ramiiieations  ol  design  changes  or  corrections  arc  not  known. 

r'lgure  4  represents  a  sceiuuio  in  whicli  tlie  svstem  under  consideration  attains  i 
mo'.ierately  high  reliability  and  tlien  stagnate'  at  tiiat  level  lor  several  pii.i'-es  i'elore 
-Mrting  ’0  improve  again.  This  sccnaiio  couM  occur  il'tlic  exact  cause  oi  ''.'tern  ;  nitiic 
;s  dillicuit  to  assess  and  would  not  be  unconmion  if  the  svstem  being  dev  eloped  .s  inghlv 
complex. 

Tigures  .5  and  h  depict  conventional  reliability  growth  patieins  one  would  expect  to 
encounter  when  evaluating  tlie  majority  of  systems.  In  Figure  5.  the  system  reliabilitv 
increases  rapidly  to  b.99  and  then  remains  constant  whereas,  in  f  igure  (>.  the  sy  tern  re¬ 
liability  increases  rapidly  to  ().90  before  becoming  constant.  It  is  important  in  tlie  eval¬ 
uation  of  the  reliability  growth  models  to  compare  their  peirorniance  in  relatively 
atypical,  although  not  uncommon,  scenarios  such  as  those  shown  in  Figures  2  through 
5  with  their  performance  in  more  conventional  situations. 

Figures  7,  8.  and  ‘■J  describe  a  constant  system  reliability  of  (i.t'O.  and  it.do  re¬ 
spectively,  While  it  would  be  rare  to  find  these  svstem  reliabilities  occurring  in  actual 
practice,  their  inclusion  will  illuminate  some  of  the  trends  of  the  grow  tit  models  under 
tlie  v.'irious  I'ailure  discounting  methodologies. 

Fach  reliability  growth  pattern  was  simulated  5'*ii  rimes  for  each  of  tlie  failure  dis¬ 
counting  combinations  descriiicd  previously.  This  will  enable  a  more  accuiato  appiais.i! 
of  tlie  capabilities  and  limitations  of  the  two  rciiabiiiiy  growtli  modeis  being  consideied. 
In  addition,  the  most  commonly  used  esiim.itor  ol  reliability,  tlie  single  phase  maximum 
likelihood  estimate,  will  be  included  on  all  figures  so  that  a  basis  li.'r  comparing  the  two 
reliability  growtli  estimators  exist.s.  I  he  single  phase  maximum  likelihood  estimaic.  in 
contrast  to  the  maximum  likelihood  estimate  with  lailure  discounting  being  evaluated  in 
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PAHERl 
INCREASING  -  CCNSV 


PATTERN  IV 

iW>ID  INCr^EASE  TO  HIGH  REUABIUTY 


Fi«iire  5.  Pattern  IV 

I 


Table  10.  PvnEPxN  IV  ISER  IN  PITS 


CALSli 

PHASE  PROItAim.I  1  Y  ('!■  SLCT'nSS 

1 

-> 

4 

5 

6 

7 

8 

■1 

10 

1 

.‘)S 

.99 

.9';s 

,99,S 

.99S 

.998 

.998 

■a 

.'.'S 

.99 

.99,T 

mi 

.99S 

.99S 

.998 

.998 

.998 

■> 

IQI 

.96 

.99 

.99.5 

i)V)S 

.99S 

.99S 

.998 

.998 

.998 

4 

.so 

.96 

.99 

.995 

.99S 

.99S 

.99S 

.998 

.998 

.998 

A 

•  t'O 

,'>0 

.99 

.995 

.''9S 

.99S 

.998 

.998 

.998 

.998 

I 
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PAHERN  V 

RAPID  INCREASE  TO  MODERATELY  HIGH  REUABILITY 


Figme  6.  Pattern 


Table  II.  PATT  ERN  V  USER  INPUTS 


^  „  PHASE  PROlJABIITTAOr  SUCCESS 

CAUSE  ' 


1 

2 

3 

4  ' 

5 

■■ 

7 

8 

■1 

10 

.‘U; 

.‘)y 

.yy 

.‘)y  j 

.yy 

.yy 

.yy 

.yy 

1 

.yy 

.yy 

mm 

.ys 

.yy 

.yy 

.yy  1 

.yy 

.yy 

.'>y  1 

.yy 

.yy 

.82 

.y6 

.ys 

.ys 

— 

.ys 

.ys 

.ys 

.'’S 

.ys 

■  SO 

.‘Ri 

.'U 

.yTs 

.y75  1 

.''>15 

WMBM 

.y75 

,o'5 

A)(y 

.yy 

.yyi  : 

.yyl 

.yyi 

.yyl 

.yyi 

.yy, 

.0(,| 

•“r 


PMIERN  VII 

CONSTANT  MODERATE  RELIABILITY 


Figure  3.  Pattern  VII 


Table  13.  PATTERN  VII  USER  INPUTS 


CAUSE 

PHASE  PRUBABIEI  rV  OF  SUCCESS 

1 

4 

5 

im 

7 

S 

u 

10 

1 

.•)v 

.7S 

.‘Kt 

.‘10 

.‘10 

.96 

.91) 

.:s 

.9') 

.7S 

.96 

.90 

.99 

.99 

.‘10 

.96 

.90 

.78 

,3 

.‘90 

.‘>9 

mm 

.78 

.9‘1 

.90 

.96 

.90 

4 

.96 

.‘10 

.9'/ 

mem 

.9(1 

.90 

.78 

.99 

.‘>0 

.‘•>6 

5 

•  VM 

.99 

.78 

.96 

.90 

.‘10 

mem 

.99 

.90 

PATTERN  VIII 
CONSTANT  LOW  REUABIU7Y 


ri 

o 
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PHASE 


Figure  9.  Pattern  \1I! 


Fable  1-1.  PATTERN  VIII  TISER  INPl.'FS 


c:ai:se 

PHASE  PROBAiUl.I  rv  OF  SI  (X  ESS 

1 

■> 

3 

4 

3 

6 

7 

■■ 

9 

10 

I 

.98 

:)5 

.s: 

•  SO 

.(H' 

.(}(> 

.80 

.82 

.9.8 

.^•>8 

■a 

•s: 

.80 

,66 

.98 

.98 

.()6 

.80 

mm 

.'<8 

Bi 

•SO 

.66 

.98 

.66 

.so 

.82 

4 

•  SO 

.06 

.98 

H 

IQI 

.98 

■■ 

.66 

.80 

5 

.'IS 

.9,‘' 

.s: 

.so 

.80 

■a 

.^>5 

.98 

.60 

this  paper,  only  considers  the  number  of  trials  and  failures  in  the  current  phase.  .Ml 
previous  data  is  discarded  since,  presumably,  the  system  reliability  tluctuates  from  plrise 
to  phase  thereby  negating  the  underlying  assumption  of  constant  reliability. 

Of  the  eight  reliability  growth  patterns  being  simulated,  only  four  (including  the 
three  cases  of  constant  system  reliability.  Figures  7.  8,  and  9)  could  have  been  produced 
with  the  simulation  as  originally  written.  Therefore,  the  addition  of  fi.\ed  phase  reliability 
will  enable  a  more  rigorous  e.xanmuuion  of  the  proposed  reliabilitv  estimators. 


IV.  EVALUATION  AND  IN TERERETATION 


This  chapter  describes  results  of  the  simulation  runs  lor  each  pattern.  1  he  per- 
I'ormance  of  the  models  are  reduced  to  graphical  form  to  facilitate  comparison.  Oniv  a 
le-.v  of  the  more  representative  runs  are  included  in  this  chapter  due  to  time  and  space 
constraints.  The  graphical  results  of  many  of  the  remaining  simulation  runs  are  in  .Ap¬ 
pendix  C  . 

1  he  analysis  is  organized  according  to  the  reliability  growth  patterns  presented  in 
the  previous  chapter.  The  performance  of  each  of  the  two  reliability  growth  models  will 
be  evaluated  with  respect  to  the  growth  pattern.  Ihe  analysis  focuses  on  how  well  the 
growth  model  tracked  Tie  actual  reliability  growth  pattern.  Included  in  this  evaluation 
is  an  examination  ot'tlie  standard  deviations  of  the  two  estimators. 

It  IS  not  the  purpose  of  this  research  to  conduct  enough  simulation  runs  so  tliat 
precise  rules  can  be  established  for  the  selection  of  discounting  parameters.  This  would 
require  analyzing  many  more  dilTerent  types  of  reliability  growth  patterns.  In  addition, 
the  choice  of  discounting  methods  and  their  respective  parameters  is  more  a  function  of 
the  particular  system  being  evaluated  than  a  function  of  the  reliability  growth  model, 
liach  system  will  have  varying  characteristics  as  will  the  personnel  responsible  for  con¬ 
ducting  the  testing  procedures. 

The  reliability  growth  patterns  will  be  referred  to  by  their  respective  Roman  nu¬ 
meral  designations.  1  hese  numerals  arc  listed  on  the  figures  describing  each  pattern  in 
the  previous  chapter  and  are  sununarized  below  for  completeness: 

1.  Pattern  1  -  Conve.xly  increasing  reliability. 

2.  Pattern  II  -  Reliability  initially  increasing  then  decreases  I'or  several  phases  before 
resuming  its  upward  trend. 

3.  Pattern  III  -  Reliability  increases  rapidly  to  approxirnatclv  O.SO.  Reliability  then 
remains  constant  for  several  phases  before  increasing  to  its  final  level  of  approxi¬ 
mately  O.du. 

4.  Pattern  IV  -  Reliability  increases  rapidly  to  approximately  0.9T. 

5.  Pattern  V  -  Reliability  increases  rapidly  to  approximately  0.90. 

6.  Pattern  VT  -  Reliability  starts  and  remains  constant  at  0,9'). 

7.  Pattern  VTI  -  Reliability  starts  and  remains  constant  at  O.OO. 

8.  Pattern  VUI  -  Reliability  starts  and  remains  constant  at  ').40. 


CONSTANT  RELIABILITIES  -  PATTERNS  VI.  VII,  AND  Mil 
1.  Standard  Discount  Metlmd 


ACTU/V.  REUABIUTY 
MLE  SINGLE  PHASE 
EXPONENTIAL  REGRESSION 


exponential  iegres''ion  moJel.  Despite  uiiing  a  reiatiNeh  ■iinail  \alue  lor  F  tLe  p'e^iiL'.eJ 
reliabilities  were  markedly  altered.  1  his  charaeteiistie  cl' the  MLI  i  1)  is  also  eMdem  "i 
the  patterns  to  lollow. 

2.  Lloyil  Discount  .Method 

The  Lloyd  discount  I'ailure  discounting  method  has  only  one  parameter,  tiie 
confidence  interval.  For  the  purposes  oL  this  «tudy  the  only  values  for  (.'I  that  were  ad¬ 
dressed  were  b.So  and  The  formula  for  appivmg  the  Tlovd  discount  method. 

Equation  2.2,  is  constructed  so  that  the  higher  the  value  of  Cl,  the  less  a  previous  faikiie 
IS  discounted.  Figure  12  graphically  represents  Pattern  \'1[I  and  the  results  of  applMiig 
the  Lloyd  method  with  Cl  =  (i.'F)  to  the  two  models.  Once  again,  the  elTect  on  the 
MLEF'L)  is  rather  dramatic  with  a  phase  .en  estimate  of  reliability  of  apj'roximatcly 
while  the  actual  system  reliability  is  1  lie  eifect  on  the  exponential  rcgres'-ion  model, 
although  signiiicant  after  piia‘.c  seven,  is  not  neaily  so  se'. ere.  Tins  marked  eifect  of 
applying  the  Lioed  discount  method  to  the  two  reiiabihty  growth  models  continued  witii 
the  other  patterns  as  will  be  seen.  .Again,  the  irnpoitant  thing  to  note  is  the  extreme 
sensitiMty  of  the  MLEFIi)  to  failure  discounting  in  general  and  to  the  chosen  s allies  of 
the  discount  parameters  in  particular. 

B.  RAPID  RELIABILITY  GROWTH  --  P.ATl  ERNS  IV  AND  V 

The  discussion  of  the  results  achicned  for  these  patterns  will  center  around  Ihittcrn 
V.  .Although  the  tendencies  of  the  two  models  were  the  same  for  both  patterns,  the 
graphical  portrayal  of  identilied  trends  is  better  observed  with  this  pattern  than  with 
Pattern  IV  where  the  final  system  reliability  approaches 

1.  Standard  Discuunt  Method 

Figure  13  depicts  Pattern  V  with  the  discount  fraction,  F.  set  at  zero.  Both  the 
MLFFD  and  the  exponential  regression  model  perform  vvell.  It  is  signiiicant  to  note. 
howcNcr.  that  the  exponential  regression  model  converged  to  the  actual  system  reliabiliiv 
much  faster  than  did  the  .MLFFD.  This  is  due  in  large  part  to  the  use  of  linear  regression 
technisiues  in  this  mo>.lel.  The  standard  deviation  of  the  exponential  regression  was 
comparable  to  the  .MLFTD  for  this  pattern  although  still  slightly  larger.  ITnh  rlie 
Mi  l  l  D  and  the  exponential  regression  model  outperformed  the  single  pliase  maximum 
likelihood  estimate  thus  lending  validity  to  the  use  of  reliability  growth  models  to  esti¬ 
mate  s\stem  reliability. 

Figure  14  shows  Pattern  V  with  F  =  0.5ft  and  I  =  3  tiials.  Ihis  high  a  rate  of 
failure  discounting  caused  the  MLFF  D  to  yield  a  phase  ten  estimate  of  o.'tqT  when  the 
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Figure  14.  Pattern  v,  F  =  and  I  =  3 

'iysrcni  relialiility  alter  the  sixth  phase  however,  it  was  mote  prollcieiu  at  matching  tiie 
actual  growtli  of  reliability  in  the  scsteni.  riicre  was  not  a  'lignilicant  Jillerencc  in  the 
standard  deviations  of  the  two  estimates. 

2.  Lloyd  Discount  Method. 

I'igure  lo  depicts  [’attorn  V  with  a  conlidence  interval  value  ol  ii.Nii.  Nmo  that, 
as  on  the  constant  system  reliability  patterns  discussed  previously,  application  ol  the 
Lloyd  discount  method  in  its  derived  I'orm  caused  both  estimators  to  predict  sssfem  re¬ 
liabilities  that  were  markedly  greater  than  the  actual  svstem  reliability.  Tins  uciid  was 
consistent  no  matter  what  values  of  Cl  were  used. 


OK? 


ri^uie  15.  Pattern  F  =  .25  and  I  =  15 


Tlie  conNcntioiial  e'^titnatc  of  reliability,  the  sinylc  phase  ina.xiinurn  likelihood 
estimate,  is  not  alFected  by  discount  procedures  since  only  cuirent  test  data  is  consid¬ 
ered.  The  discounting  methods  addressed  in  this  paper  are  only  aisplied  to  failures  oc¬ 
curring  in  previous  phases. 

C.  DFCRE.XSING  RFLI.VBILFr  V  -  P.\  ITER.N  11 
1.  Standard  Discount  Metliud 

I  igurc  1~  shows  Pattern  II  with  no  fiilurc  discounting  applied.  Of  special  in¬ 
terest  in  this  graph  is  the  performance  of  the  two  rclialMlity  growth  models  duiing  the 
period  ol  declining  relialhlity.  Ihc  e.xponcntial  regression  model  was  the  only  moilel  that 
actually  demonstrated  a  decrease  in  reliabilitv  althouch  it  still  overestimated  the  actual 
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PAHERN  II  -  NO  DISCOUNTING 


- * - ACTUAL  REL'ABILIPT 

— a —  MLE  SINGLE  PH/>5E 

—  7 —  EXPONENTIAL  REG.TESSION 


Figure  17.  Pattern  II.  No  Discounting 

The  standard  deviation  of  the  exponential  regression  estimate  still  exceeded  that 
of  the  MLEiFD.  This  characteristic  is  fairly  consistent  for  all  the  patterns  iIku  were  ex¬ 
amined  in  the  course  of  evaluating  these  two  models.  It  should  he  noted,  however,  that 
a  small  standard  deviation  does  little  good  if  the  estimator  is  yielding  markedly  incorrect 
predictions. 

2.  Lloyd  Discount  Method 

Figure  19  shows  Pattern  II  with  Cl  =  d.9i).  1  he  application  of  the  Floyd 
method  profoundly  alfected  the  performance  of  the  MFFF'D  while  only  slightly  altering 
the  exponential  regression  model.  This  method  of  failure  discounting  did  cause  both 
models  to  overestimati  system  reliability  by  phase  nine  but.  again,  had  less  of  an  effect 
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l  igure  IS.  Pattern  II.  F  =  .75,  I  =  6 

on  the  exponential  regression  model.  It  is  especially  noteworthy  to  observe  tlie  eflect  of 
the  Lloyd  method  on  the  MI.FiFD  during  the  period  ordecrcasing  system  reliability.  The 
real  sensitivity  of  the  .MLCLD  to  discount  methods  and  parameters  is  particularly  evi¬ 
dent  in  this  pattern. 

0.  IiNI  ERMi  n  AN  I  RFLIARILITY  GROWTH  -  PATTERN  III 
1.  Standard  Discount  Method 

r  igure  Z<)  ilepicts  the  performance  of  the  two  models  wlien  no  failure  dis¬ 
counting  is  applied.  The  exponential  regression  model  outperformed  the  Ml.LTl)  at  ev¬ 
ery  phase.  .Again,  the  exponential  regression  model  succeeded  in  capturing  the  trend  of 


fe; 


the  reliability  growth  pattern  from  phase  lour  to  phase  six  whereas  the  MLhTD  exhib¬ 
ited  a  constant  rate  of  increase. 


\\  lien  the  discount  parameters  I'  and  I  were  set  at  0.25  and  six  respectively,  the 
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.Ml.EFD  did  much  better  as  can  be  seen  in  rigure  21.  Both  models  converged  to  the 
correct  prediction  of  system  reliability  by  phase  ten.  However,  the  exponential  regression 
model  converged  to  the  correct  value  much  moie  quickly  than  did  the  MLl'.l'D.  I  lie 
standard  deviations  ol'  the  two  estimates  were  not  too  signilicantly  dilTerent  Irom  each 
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other  lor  this  particular  pattern  and  discount  method. 

'I  he  perlbrmance  of  the  two  n.odels  on  reliability  growth  patterns  11  and  III 
demonstrate  one  of  the  inherent  advantages  of  the  exponential  regression  methodology. 
Since  this  method  makes  use  of  the  tcchni({ues  ol'  linear  regression,  it  i.s  better  able  to 


track  changing  system  reliability  than  is  the  MLHFD.  I  his  attribute  of  the  exponential 
regression  model  will  be  developed  further  in  the  next  chapter. 

2.  Lloyd  Discount  Method 

Figure  22  shows  how  the  models  performed  using  Lloyd  failure  discounting 
with  Cl  =  0.90.  Again,  applying  the  Lloyd  method  caused  the  MLLI'D  to  overestimate 
system  reliability  after  phase  lour.  Some  slight  overestimation  did  occur  in  the  expo¬ 
nential  regression  model  during  phases  nine  and  ten  however,  the  overall  cll'ect  was  to 
enable  tlie  model  to  better  track  actual  system  reliability  during  the  middle  phases. 
During  the  discussion  of  the  discounting  methodologies  in  Chapter  II.  it  was  noted  that 
the  Lloyd  failure  discount  method  is  applied  after  every  trial  that  did  not  result  in  the 
reoccurrence  of  a  previous  failure  cause.  This  means  of  application  is  a  likely  source  of 
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Figure  21.  Pattern  III,  F  =  .25.  I  =  6 


the  overestiniation  characteristics  of  both  models  when  Lloyd  discounting  is  applied. 
An  idea  on  a  possible  remedy  to  this  problem  will  be  presented  at  the  end  of  this  chapter. 

E.  COiNVEX  RELIABILITY  GROWTH  -  PAM  ER.N  I 

.Most  reliability  growth  patterns  arc  thought  to  be  essentially  concave  in  shape, 
much  like  Patterns  IV'  and  V.  This  is  normally  a  valid  .assumption  in  that  major  im¬ 
provements  in  the  reliability  of  a  new  system  most  often  occur  duiing  the  early  stages 
of  development  as  the  obvious  Haws  in  the  design  and  manul'acture  become  apparent. 
It  is  not  inconceivable,  however,  to  experietice  a  period  of  convex  growth  such  as  shown 
in  Pattern  I.  One  would  want  to  have  confidence  in  the  chosen  reliability  estimator  to 
correctly  estimate  system  reliability  resardlcss  of  its  actual  form. 


I.  Standard  Discount  Mctliod 

Figure  23  sliows  the  performance  of  the  two  models  witli  no  failure  discounting 
being  applied.  It  is  apparent  that,  even  without  the  benefit  of  failure  discounting  tcch- 
nii-iues,  the  exponential  regression  model  does  well.  It  is  able  to  relied  the  trend  of  actual 
system  reliability  growth  and  the  actual  estimates  are  laiily  close  to  the  true  system  re¬ 
liability.  The  .\ILEFD  does  not  do  so  well  and.  in  fact,  is  e\en  outpcrlorined  bv  the 
single  phase  maximum  likelihood  estimate  in  tire  later  phases.  One  would  expect  in  this 
case  to  have  significant  i exults  wlien  failure  discounting  is  applied  due  to  the  demon¬ 
strated  sensitivity  of  the  MLLi  I)  to  this  procedure. 
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PATTERN  I  -  NO  DISCOUNtING 


Figure  23.  Pattern  I.  iNo  Di^icounting 

Figure  24  depicts  Pattern  I  with  I'  =  .75  ami  I  =  (i.  Altliougli  Failing  to  captuie 
the  sliape  of  the  actual  reliability  pattern,  the  pliase  estimates  produced  by  the  MLiil  l) 
are  quite  good.  Fractionally  removing  75  percent  ofa  previous  failure  after  si\  successful 
tiials  is  a  ..ulrstantial  discount  and  one  would  expect  that  such  a  large  discounting 
scheme  would  lend  itself  more  to  the  comcntional  concave  reliability  growth  patterns. 
This  particular  tvpe  of  scenario  would  benelit  greatlv  by  the  ability  to  alter  the  discount 
paiameters  during  the  course  of  the  test.  Initially,  one  would  nut  want  to  discount  pre¬ 
vious  failures  by  a  large  amount  since  actual  s\stem  reliability  is  only  improving  mai- 
ginally.  .As  the  system  evoKes  and  reliability  begins  to  make  substantial  jumps  then 
previous  failures  sliould  be  discounted  more.  Of  course,  this  presumes  a  knowledge  of 
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Figure  24.  P.nltern  I.  F  =  .75  and  I  =  6 

the  actual  '.ystem  reliability  in  order  to  properly  cic'.ign  sucli  a  plan  and  this  is  never  the 
case,  lloweser.  in  depth  knowledge  of  tlie  proposed  system  could  indicate  the  exiicclcu 
duipe  of  the  relitihihty  pattern  and  tins  could  be  made  use  of  in  test  design.  .Althouch 
changing  the  discount  parameters  during  the  course  of  the  tect  is  possible  in  actual 
practice  it  is  not.  uni'ortiinately.  possible  to  do  with  die  simulation  m  its  current  lorm. 
1  lie  parameters  lor  the  discount  procedures  remain  constant  once  they  are  input  at  the 
beginning  of  the  simulation. 

2.  Llosd  Discniiiit  .Method 

Figure  25  shows  the  Lloyd  method  with  FI  =  n.'X*.  The  e.xponcntial  regression 
model  maintains  the  shape  of  the  true  system  reliability  while  conversinc  to  the  actual 
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Figure  25.  Pattern  I,  Lloyd  method.  Cl  =  .9 

reliability  by  phase  eight.  The  MLLFD  becomes  concave  in  shape  .nncli  as  it  did  lor  tlie 
standard  discount  nictliod  shown  in  Figure  24.  However,  tlic  imennediate  phase  rcli- 
al.nlity  estimates  arc  very  good.  I  he  standard  deviation  o(  tlie  MLFI  D  continues  to  be 
less  than  that  of  the  exponential  regression  model  aitliough.  as  shown  in  Chapter  II.  tlie 
moan  square  error  for  the  MI.FFD  exceeds  that  of  the  exponential  regression  model. 

The  Lloyd  method  does  better  with  this  pattern  than  with  any  other.  Previ¬ 
ously.  application  of  this  method  caused  both  models  to  overestimate  system  reliability 
quite  significantly.  In  this  case,  using  this  method  improved  the  performance  ol  the  ex¬ 
ponential  regression  model  and.  although  distorting  the  shape,  resulted  in  improved  re¬ 
liability  estimates  from  the  MLEFD  as  well. 
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F.  LL(>\ D  FAILI  RE  DISCUI  MIM,  (  UFAl'NlIf  O  I 


.'.ppiMilg  'ii';  I.io'.d  Ijiluie  .'d  :.i  .[  ■  .  !;:i  ^  '-■c'!;'. 

.auscd  notii  laodcN  to  o'.cicMiiikite  t!iC  trjc  s'v'-t.::!  ’  i  '  '.t  ;■  :  i  ' 


M\ci.!  ’^'na'^c  reluiHilii v  'jro'.'.th  pattern''  ti.  it  '.'rerc  ui  iies'od  n; 


^  i  j  1 '» 


J''  ’  L\ 


e;  va'ii  patterns  :l;at  '.vere  anal'./ed  m  ('apt.nn  Dial.e  '  -aoin  pac'. '  on 

One  potential  source  of  this  probicin  is  tiiat  the  l  losd  nietlu,d  .s  .n'piied.  arer 
j'.  erv  '.ue^esstul  trial. 

1  he  lornuila  for  computing  adjusted  failures  usine  the  Ido'.d  method  is  ^•'.en  In 
Fquation  2.2  repeated  iselow; 


ADJISTED  I'AILURE  =  I  -  ( 1  -  C7i  T  ,r  E  >  0. 


II  1  equals  zero  then  the  adjusted  failure  value  is  set  'o  one.  I '.ns  lormula  is  used  to 
s  MUPUie  adjusted  I'liluies  after  each  tii.il  in  '.‘.  Inch  tlie  c.iiise  for  a  jne'.  lutis  [hiluie  .iid 
not  reoscur.  Siiioe  th.e  prol'lem  appears  to  lie  m  the  number  ol  times  "he  nietiiod  is  aj-- 
plied  tile  inclusion  of  an  interval  may  lead  to  more  accurate  results, 

.A  modified  version  of  the  Lloyd  discount  method  which  employs  an  interval  of 
ai’idication  is  as  follows: 


Alin  SEED  EAILERE  =  1  -  (1  -  CFlTT  ,f  M  >  n 
whore,  in  this  case. 


(4.1) 


M  =  /.V/( 


T 


l.Dl 


I.DI  is  defined  as  the  I.loyd  Discount  Interval  and  becomes  another  parameter  that  can 
be  altered  by  the  user  of  the  simulation.  If  M  equals  zero  tlien  the  adjusted  failure  value 
IS  set  at  one  as  before.  By  its  construction,  one  can  observe  that  for  ail  values  of  T  Ic.ss 
than  the  prcdesignalcd  LDl.  tlien  the  value  ol  M  will  be  zero.  M  will  increase  bv  one  for 
each  group  of  successful  trials  meeting  the  l.Dl.  Note  also  that  specil'ving  an  I,DI  ol  one 
allows  the  user  of  the  simulation  to  employ  the  I.loyd  method  in  its  original  form. 

Some  preliminary  runs  of  tlie  simulation  were  made  with  this  modilicaiion  included. 
I  ho  results  were  encouraging.  Figure  12  examined  previously  illustrated  the  ellcct  o(  t!.e 
Lloyd  method  on  the  two  models  when  svstem  reliability  was  actuallv  a  comtaiu  0.411. 
I  igure  26  shows  the  result  of  applvmg  this  modilied  version  of  the  Lloyd  method  with 
(.he  value  for  Cl  remaining  at  (J.60  and  the  LDI  set  at  eight.  Both  the  .\1LLI  D  and  tlie 
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Figure  26.  Pattern  Modified  Lloyd  Method.  Cl  =  .9.  LDI  =  8 


exponential  regression  model  performed  much  better  than  before.  I  lie  LDI  value  of 
eiglit  resulted  in  similar  improvements  for  the  other  constant  system  reliability  patterns. 

The  results  on  the  other  patterns  were  similar  to  those  lepresented  in  Figure  17  but 
the  \alue  for  the  LT)I  had  to  be  adjusted  for  each  pattern.  Before  any  specific  conclu¬ 
sions  t  an  be  drawn  about  the  benefits  of  this  modification  many  more  computer  runs 
and  more  in  depth  analysis  is  rcsjuired.  However,  based  on  these  prcliminaiy  results,  the 
inclusion  of  an  interval  does  appear  to  have  some  merit. 


*  •  *  • 
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V.  SUMMARY,  CONCLLSIONS  AND  RECOMMENUA  1  lO'sS 
A.  SLMMARV 

The  purpose  of  this  paper  was  to  evaluate  two  diseiete  leliability  eiouth  iiiouci'-. 
I'aclt  of  f.hese  models  has  tlie  ability  to  U'^e  fractioiiaHy  Ji^eininted  lailure  liaia  to  ma^e 
an  estimate  ot'  the  actual  system  rehabiiity  at  any  point  durme  the  tonautt  o:  a  te^'t. 
Fractionally  discounting  failures  occurring  earl>  in  a  test  procedure  allow>.  •lie  daia  ac¬ 
cumulated  during  the  initial  stages  of  a  test  to  be  used  in  current  reiia'nht'^  c^tmiatc'.. 

1  herefore.  if  the  discounting  procedure  is  performed  wisei\.  nime  accurate  rchabiiits 
estimates  witli  a  higher  degree  of  confidence  should  reculi. 

Two  procedures  for  discounting  presious  system  failuics  were  dssciibed  in  <  liap'cr 
II.  I  lie  standard  discount  metltud  requires  the  user  to  speens  two  o  ;:am':'crs.  the 
tion  of  failure  to  be  removed  and  the  discount  interval.  1  he  iraction  oi  .’.uhiie  tc  i'C  ic- 
moved,  relerred  to  as  the  discount  fraction  or  F.  specilies  how  much  a  piesiom.  failute 
is  reduced  upon  each  application  of  the  discount  procedure.  I  lie  discount  mtersai.  I,  o- 
simply  the  number  of  successful  trials  (trials  without  a  reoccurience  of  the  particular 
lailure  cause)  that  must  occur  between  applications  of  the  procedure.  I  he  other  lailure 
discounting  method  was  proposed  liy  David  Llotd  and  is  referred  to  as  the  Fiend  dis¬ 
count  method.  This  method  utilizes  only  one  input  parameter,  the  Iccc!  ol' coiilidciicc 
lor  an  upper  confidence  interval  on  reliability.  Tliis  method,  in  its  original  lorm.  is  ap¬ 
plied  after  every  succe.s.sful  trial.  .A  niodilied  Lloyd  discounting  metliod  wliich  ei.iplovs 
an  interval  of  application  was  derived  in  Chapter  IV. 

Fwo  discrete  reliability  growth  models  were  desenised.  The  fust  model  is  a  form  of 
tlie  ma.xiinum  likelihood  estimate  with  the  addition  of  failure  discounting.  Since  the 
ma.ximuin  likelihood  estimate  assumes  a  constant  reliability  between  phases,  tlie  addition 
of  failure  discounting  will  enable  this  model  to  estimate  changing  system  reliability,  flie 
Maximum  Likelihood  Estimate  with  Failure  Discounting,  or  .MLF.FD,  estimates  system 
reliability  as  the  ratio  of  tlie  number  of  successful  trials  to  tlie  total  number  oftiials  with 
this  ratio  being  computed  after  application  of  failuie  discounting. 

The  other  model  was  developed  by  If.  ChernofFand  W.M.  Woods  and  is  a  deriva¬ 
tion  of  an  exponential  single  phase  icliability  estimate.  1  his  model  employs  linear  re¬ 
gression  techniques  to  estimate  reliability  after  any  cliange  has  been  made  to  the  unit 
of  hardware  under  test.  Because  of  this,  the  exponential  regression  model  is  able  to  track 


changing  sc'tcnn  rciiabiiii}  '.vihioiu  the  benciit  oi' lahare  hiscountnig  aithong.i  bic  u^c 
ol  [ailnre  Jncounting  can  enhance  t!ie  inoJei  s  performance. 

fl'.e  jiieans  oi' evainatmg  thc'^e  two  inoUeis  was  a  Monte  t'arlo  '-inahaia-n.  li.o 
simulation  u<eh  was  originally  written  by  Captain  .lames  Drake  ami  vcas  mchilieh  'O 
fi\ed  piia^e  reliabiiities  could  be  modeled.  I  he  fixed  phase  reliability  mudnuaiion  ^ 
the  user  of  the  simulation  to  evaluate  the  perfoirnancc  of  the  two  reliabilitv  growih 
models  on  any  pattern  of  actual  reiiaohity  growth  desired.  Tlius.  periods  of  de.aning 
reliability,  peiiods  of  no  reliability  growtii.  and  convexlv  growing  reliability  among  oth¬ 
ers  can  be  easily  modeled  and  the  perlbrinance  of  the  two  growth  models  anaK  /ed. 

L'igiu  varying  reliability  growtlt  patterns  were  analyzed.  Botli  of  the  reliability 
grou  til  models  ucrc  evaluated  against  these  eight  patterns  and  under  13  diJl'ercnt  lailure 
discounting  combinations  including  no  I'aiiure  discounting.  Ifach  model,  pattern  and 
failure  discounting  combination  was  replicated  onu  mnes,  1  he  ability  oi  each  mcCe!  'o 
accuiately  track  the  actual  svstem  fchabiiitv  was  e'.aiuaied  ^long  with  an  exaiuination 
of  the  stability  of  the  estimate  produced  by  tlie  model. 

U.  C0l^CLlS10^S 

The  evaluation  of  the  two  reliability  growth  models  on  the  eight  diUcrcnt  rcliabilny 
growth  patterns  leads  to  sc. eral  general  conclusions.  The  most  obxious  conclusion  is 
that  tlie  use  of  citlicr  one  of  these  growth  models  is  superior  to  the  standard  single  pliase 
maximum  likelihood  estimate.  Both  growth  models  were  fill  more  accurate  in  tcims  cf 
estimating  the  true  system  reliability  and  the  \aiumco  associated  with  the  model  esti¬ 
mates  was  less  tlian  that  produced  by  the  single  phase  .\I!,T.. 

fhe  .MLEI'D  is  particularly  sensitive  to  the  choice  ol  failure  discounting  pio>-edurc 
and  parameters.  I  his  sensitivuy  is  made  use  of  in  oidcr  to  allow  the  model  to  track 
changing  reliability.  However,  this  property  also  places  a  premium  on  the  aisiluy  ol  the 
test  engineer  to  correctly  select  the  proper  Jiseouiuing  method  and  p.u  imeicrs. 

fhe  .MLETD  did  very  well  when  the  actual  system  reliability  was  constant,  (fon- 
stant  system  reliability  is  seldom  present  m  actual  rciiabilitv  testing,  however.  I  he 
Ml. LI  D  also  did  well  when  the  true  reliability  was  iiwreasing  ,it  a  constant  rale.  .\l- 
tliough  the  exponential  regression  model  more  closely  followed  the  actual  rcliabiliix 
growth  pattern,  the  MLf'KD  would  be  picferred  in  this  instance  because  of  its  smaller 
V  iriabilitv.  fhe  Ml.Iil'D  did  best  with  these  pattcins  when  the  standard  i.!iSi.ouut 
method  was  used  with  the  discount  fraction  equal  to  0.25  and  the  discount  interval  equal 
to  6  or  15. 
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liio  exponential  reerc'^sion  moJel.  altlv.jueii  moic  ai:ai\e  a  -l.e  eaix  '.•.ciees  a 
te^tine.  Jcrnor;''trareLl  tlic  ab:iit>  to  aecuraiel'.-  rrack  each  riie  aieiit  lei  aiaia'.  e:.'  \  '  i 
patterns  evaluated,  even  without  tiie  use  oi’tailuie  Ji-.eountine.  Thi-.  urulei  ;•;  alw)  lau^ii 
less  sensitive  to  the  choice  of  discount  method  aitlioueh  its  oei ;ai nian.e  can  :se  saaic- 
'.'■hat  eiilianced  tiiroue.h  I'ailure  discounting.  I'lie  MI-Ll’l),  on  the  oihei  liaiivi.  l.iiled  to 
track  tile  decline  in  reliability  in  Pattern  II. 

rncreidre.  the  exponential  regressiun  mode!  is  recoinniended  ;or  Mtuaiu-ns  '.'.lieic 
a  proven  means  olTailure  discounting  does  not  exist.  IT  one  is  coiiiident  of  tire  I'aiiure 
discounting  inethodulogy  tlien  the  MLlif  D  may  be  more  advantageous  due  lo  the  sta¬ 
bility  of  tlie  estimates  produced  by  it.  Regardless  ol' the  degree  orconlisience  one  has  in 
the  lailure  discounting  procedures,  the  exponential  regiession  inoilcl  is  recommended  il 
abnormal  reliability  growth  is  anticipated.  .Abnormal  reliability  guwvth  can  be  delink'd 
as  1  reliability  pattern  that  has  periods  of  vlecreasing  or  coiistani  leliabiliiv  in  die  niidd'c 
Ol  a  generxiilv  increasing  grou  th  late.  1  he  exponential  rcgiessiun  mouci  dicniunsti aic  i 
superiority  to  the  MbbI  D  in  its  ability  to  track  dilleiing  patterns  of  growth. 

I'iie  Lloyd  discount  method,  in  its  original  Ibrin,  caused  both  models  to  overesti¬ 
mate  the  actual  system  reliability.  Therelbre,  use  ol'this  method  witli  these  two  reliability 
growtli  models  is  not  reconunended.  A  modified  version  of  the  Lloyd  method  winch 
employs  a  discount  interval  demonstrated  superior  perl'oimcmcc  in  some  patterns  to  the 
original  Lloyd  discounting  method. 

C.  RECOMME^DAT10^S  FOR  FLRTHERSrtDY 

The  following  are  recommendations  for  I'urthcr  study  and  possible  improvement  to 
the  models  introduced  in  the  study: 

•  This  study  advlressed  only  discrete  reliability  growth  inodcLs.  A  similar  s[ud>  aim- 
ly/ing  the  characteristics  of  continuous  reliability  growth  models  siiould  be  con¬ 
ducted.  fhis  would  necessitate  the  construction  of  a  din'erem  simulation. 

•  C'urreiuly.  the  failure  discounting  mechanism  used  in  the  simulation  employed  in 
this  analysis  only  allows  tlie  user  to  set  parameter  values  at  the  beginning.  This 
should  be  mculilied  so  that  discount  parameters  can  be  changed  during  tlie  course 
of  the  simulation. 

•  T  he  I'aiUire  discounting  mcvlianism  should  be  modified  so  that  dilferent  parameters 
can  be  a[-'plied  to  dillerent  failuie  causes.  It  may  be  desired  that  svsteni  failures 
caused  by  failure  cause  ,\,  for  example,  be  disc<Hmted  loss  than  those  caused  by 
failure  cause  IT  1  his  is  not  possible  witli  the  current  simulation. 

•  Lurther  analysis  is  required  of  the  modified  Lloyd  failure  discounting  method. 
Many  more  simulation  runs  with  varying  reliability  growth  patterns  and  dilfcicm 


conibinations  ol  uiscomu  parainetors  are  required  before  any  colid  LoncliiMCii'- 
bc  reuLned. 

•  A  '.veielited  least  '■qaares  inctlioJ  that  weiehte  leccnt  datti  more  hea. liv  t'iaii  i 
vious  data  should  be  analyzed  to  determine  us  abiliiy  to  track  reiiabiiiiy  ero 
patterns  v.  itli  suciden  ciuinees  in  slo['e  or  direction. 


ArrCNDlX  A.  derivation  of  rHE  E?;PONENTlAL  REGRESSION 

MODEL 


Tins  model  was  developed  bv  H.  Cheinolfaiid  Woods.  Tlie  derivation  has 

been  presented  in  previous  works,  most  recently  by  W.M.  Woods  in  a  paper  entitled 
'■Reliability  Growth  .Models'  and  by  Captain  James  Drake  in  a  thesis  entitled  "Dis(.rete 
Reliability  Growth  Models  using  Failure  Discounting".  The  exponential  regression 
model  is  based  on  the  exponential  single  phase  reliability  estimate.  The  exponential  sin¬ 
gle  phase  estimate  may  be  expressed  mathematically  as  Ibllows: 

:,■»  _  1 

=  J  —  t’ 

'I'.ie  exponential  regression  model  uses  linear  regiession  to  estimate  .A  so 
A  =  y.  ~  jik 

where  k  is  the  testing  phase  being  used  to  compute  system  reliability,  '['hus.  the  expo¬ 
nential  regression  model  has  the  following  formula: 

R  =  1  - 

The  estimates,  y.^  and  /!;.  for  y  and  /?  at  the  end  of  the  kth  phase  are  obtained  using 
the  techniques  of  linear  regression  and  an  unbiased  estimate  for  (a  +  /1/ci  .  Let  F  denote 
the  total  number  of  failures  possible  during  the  kth  phase  and  let  j  equal  the  failure 

number  in  phase  k  such  that  j=  1.2,3 . /' ,  .V.  equals  the  number  of  trials  between  the 

(j-Dst  failure  and  the  jth  failure,  including  the  jth  I'ailure.  in  the  kth  phase. 

■An  unbiased  estimate  of  (:<  4-  jik)  using  the  jth  set  of  trials  in  phase  k  is  given  by: 

} ;  .  =  1  -f  -b  ...  +  Y  I  (/  •Vfe  >  2 

If  .V.i  =  1  (the  first  test  was  a  failure)  then  F*  is  set  equal  to  zero.  I'lie  least  squares 
estimates,  y.  and  /L,  for  y  and  /t  at  the  kth  phase  are  then: 


a 

'<1 


}  =(},  +  ?,  +  ...+  y,)//< 

Substituting  these  estimates  for  and  /?*  into  tlie  reliability  eciuation  yields  the  Ibllowin 
estimate  of  reliability  for  phase  k: 

,  =  1  _  +  /,  >  1 
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THE  FOLLOWING  EXTERNAL  FILES  ARE  USED  BY  THE  PROGRAM 

INPUT  :  DATA  AND  PARAMETER  INPUT  FILE  (DEVICE  if  10) 

THESIS  :  OUTPUT  FILE  CONTAINING  INTERMEDIATE  COMPUTATIONS 
(DEVICE  if  20) 

RELIAB;  OUTPUT  FILE  CONTAINING  FINAL  RESULTS  OF  THE  SIMULATION 
'DEVICE  -i  30) 

EST  :  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 
REPLICATION  OF  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 
(DEVICE  =f  40) 

MLEWD  ;  OUTPUT  FILE  CONTAINING  MLE  ESTIMATES  USING  DISCOUNTING 
FOR  EACH  PHASE  AND  EACH  REPLICATION 
(DEVICE  if  50) 

MLESP  :  OUTPUT  FILE  CONTAINING  MLE  ESTIMATE  FOR  EACH  SINGLE  PHASE 
AND  ALL  REPLICATIONS  USING  NO  DISCOUNTING 
(DEVICE  if  60) 

REGEST  ;  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 
REPLICATION  OF  THE  EXPONENTIAL  REGRESSION  ESTIMATE 
(DEVICE  if  70) 

THE  FOLLOWING  IS  A  LIST  OF  KEY  ARRAYS  USED  IN  THE  SIMULATION 

A  :  MAIN  WORKING  ARRAY  CONTAINS  PROBABILITY  OF  SUCCESS  FOR 

EACH  FAILURE  CAUSE,  NUMBER  OF  TRIALS  UNTIL  FAILURE  FOR 
EACH  FAILURE  CAUSE  AND  THE  SYSTEM,  CAUSE  OF  FAILURE, 

PHASE  NUMBER,  ADJUSTED  NUMBER  OF  TRIALS  AND  ADJUSTED 
NUMBER  OF  FAILURES 

DIMENSION  (  ((2*#CAUSES)+7) , (/FAILURES  ) 

NFAPH  :  CONTAINS  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE 
DIMENSION  (1, (/PHASES) 

NFCAUS  :  BINARY  ARRAY  USED  TO  DETERMINE  IF  A  FAILURE  OCCURRED  IN 
A  PHASE 

DIMENSION  (  1,^/FAILURE  CAUSES) 

NTRIAL  :  CONTAINS  THE  NUMBER  OF  TRIALS  SINCE  LAST  FAILURE  OR 
DISCOUNTING  FOR  EACH  FAILURE  CAUSE 
DIMENSION  (  1.. -/FAILURE  CAUSES  ) 

PUREST  :  RECORDS  THE  PHASE  ESTIMATE  FOR  EACH  ESTIMATOR  WITHIN  A 
SINGLE  REPLICATION 
DIMENSION  (4, (/PHASES) 

ROW  1  ;  WOODS  WEIGHTED  AVERAGE  ESTIMATE 
ROW  2  :  MLE  WITH  DISCOUNTING 


ROW  3  :  SINGLE  PPASE  MLE 

>>  ROW  4  ;  EXPONENTIAL  REGRESSION  ESTIMATE 

*  AREL  ;  CONTAINS  nCTUAL  SYSTEM  RELIABILITY  IN  EACH  PHASE 
DIMENSION  ( 1,?:-THASES) 

='<•  YJK  :  CONTAINS  YJK  VALUES  UP  TO  1000 


DIMENSION  f  1,1000) 

CUMSF  :  CONTAINS  THE  NUMBER  OF  SUCCESS  AND  FAILURES  FOR  EACH 
FAILURE  CAUSE  (USED  WITH  WOODS  WEIGHTED  AVERAGE  EST.  ) 

*  DIMENSION  (3, //FAILURE  CAUSES) 

*  ROW  1  :  NUMBER  OF  FAILURES 
ROW  2  ;  NUMBER  OF  SUCCESSES 

ROW  3  :  ADJUSTED  NUMBER  OF  SUCCESSES  * 

*  REG  :  ARRAY  USED  TO  COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

DIMENSION  (5, //PHASES) 


* 

ROW 

1 

K 

BAR 

•V 

* 

ROW 

2 

Y 

BAR 

* 

* 

ROW 

3 

Y 

BAR  FOR  THE  PHASE 

* 

* 

ROW 

4 

B 

HAT 

■sV 

* 

ROW 

5 

A 

HAT 

‘V 

THE  REMAINING  ARRAYS  ARE  USED  TO  COMPUTE  THE  MEAN  AND  VARIANCE 
OF  EACH  ESTIMATE  AT  EACH  PHASE.  THEY  ALL  HAVE  THE  SAME  DIMENSION'S 
i\ND  STRUCTURE 


Vf 

DIMENSION  (4,?/PHASES) 

* 

Vf 

RCW  1 

RUNNING  SUM  OF  ESTIMATES 

-.v 

Vf 

ROW  2 

RUNNING  SUM  OF  SQUARED  ESTIMATES 

V.' 

ROW  3 

MEAN  OF  THE  ESTIMATES 

t'c 

* 

Vr 

ROW  4 

STANDARD  DEVIATION  OF  THE  ESTIMATES 

* 

•V 

Vt 

EST 

VALUES 

FOR  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

Vc 

iV 

MLEWD 

VALUES 

FOR  THE  MLE  WITH  DISCOUNTING 

Vf 

Vf 

MLESP 

VALUES 

FOR  THE  SINGLE  PHASE  MLE 

Vf 

REGEST 

VALUES 

FOR  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

C  DEFINE  AND  DIMENSION  VARIABLES 


PARAMETER  ( NR=50 ,NC=200 ) 

INTEGER  REP , CUMSF , DISOPT .FRELOP , LDI , ALD 
REAL'>4  MIN 

REAL-'- 8  DSEED ,  MLESP ,  MLEWD ,  EST ,  EUL 

DIMENSION  NFAPHCNR) ,A(NR,NC) ,NPCAUS(NR) ,NTRIAL(NR) ,PHREST(4,NR) .ES 
CT(4,NR)  ,MLEWD(4,.NR)  ,MLESF(4,NR)  ,REGCST(4,NR)  .ARELCNR)  ,YJK(  1000)  ,CU 
CMSF(3,NR) ,REG(5,NR) 

C  READ  IN  THE  NUMBER  OF  CAUSES  TO  BE  USED  (  NCAUSE  )  AND  THE  NUMBER 
C  OF  PHASES  (  NPHASE  )  IN  THE  TEST 

READCIO,*)  NCAUSE 
READCIO,*)  NPHASE 


C  CHECK  IF  FIXED  RELIABILITY  OPTION  IS  DESIRED.  FIX  EULER’S  NUMBER. 


READCIO,-^)  FRELOP 
EL'L  =  0.  5  7 72 1566A8 

C  CREATE  VARIABLES  FOR  THE  ROW  INDICES  OF  THE  WORKING  MATRIX  (  A  ) 

C  IPHASE:  PHASE 

C  ISYSPR;  ACTUAL  COMPONENT  RELIABILITY 

C  INTR:  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 
C  IFAILC:  CAUSE  OF  THE  FAILURE 

C  lADJF;  ADJUSTED  NUMBER  OF  FAILURES  E 

C  AFTER  DISCCUNTING  HAS  BEEN  APPLIED 

C  lADJT:  ADJUSTED  NUMBER  OF  TRIALS  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 
C  lYJK:  YJK  COMPUTED  ON  THE  ADJUSTED  NUMBER  OF  TRIALS 

IPHASE  =  (2’'NCAUSE)  +  1 
ISYSPR  =  IPHASE  +1 
INTR  =  ISYSPR  +  1 
IFAILC  =  INTR  +  1 
lADJF  =  IFAILC  +  1 
lADJT  =  lADJF  +  1 
lYJK  =  lADJT  +  1 


C  READ  IN  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE  (  NFAP}[(  I )  )  AND 
C  COMPUTE  THE  TOTAL  NUMBER  OF  FAILURES  IN  THE  TEST  (  NFAIL  ) 

NFAIL  =  0 

DO  10  I=1,NPHASE 

READdO,*)  NFAPH(I) 

NFAIL  =  NFAIL  +  NFAFH(I) 

10  CONTINUE 

C  INPUT  THE  FROBABILm  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C  IN  EACH  PHASE  IF  FRELOP  EQUALS  ONE. 

IF  (FRELOP  .EQ.  ll  THEN 
DO  15  I=1,NCAUSE 
DO  15  J=l. NFAIL 
READdO,*)  A(I,J1 
15  CONTINUE 
ELSE 

C  INPUT  THE  PROBABILITY  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C  IN  THE  FIRST  PHASE  IF  FRELOP  EQUALS  ZERO. 

DO  20  I=1,NCAUSE 

READ( 10,*)  A(I,1) 

20  CONTINUE 

END  IF 

C  INPUT  THE  REMAINING  VARIABLES  ,  THE  NUMBER  OF  SUCCESSFUL  TRIALS 
C  BEFORE  A  DISCOUNT  IS  APPLIED  (N);  THE  DISCOUNT  FACTOR  (R);  THE  SEED 
C  FOR  THE  RANDOM  NUMBER  GENERATOR,  GGUBFS,  (DSEED);  RELIABILITY 
C  GROWTH  FRACTION  (FRIMP);  TRIGGER  FOR  PRINTING  INTERMEDIATE  OUTPUT 
U  (lOPT) 


3 


I 


TRIGGERS  FOR  SAVIN’S  F.ACIf  ES-^IMATE  AT  EACH  PHASE  FOR  EACH  ESTIMATCK 
IjPTI  ;  WOODS  WEIGHTED  AVERAGE  MODEL 
IGt'T2  :  MLE  WITH  DISCOUNTING 
IOPT3  :  SINGLE  PHASE  MLE 
I0PT4  :  EXPONENTIAL  REGRESSION  MODEL 
DISCOUNTING  OPTION  TRIGGER  (BISOPT');  LLC'i’D  FAILURE  DISCOUNTING 
PARAMETER  (GAMMA);  LLO'iD  DISCOUNT  INTERVAL 


READ( 

READ( 

READ! 

READ( 

READ( 

READ( 

READ( 

READ( 

READ( 

READ( 

READ( 

READ( 

READ( 


10,-) 
10  .*) 
10,*) 
10,*) 
10.*) 
10,*) 
10,*) 
10.*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 


R 

LSEEB 

FRIMP 

NREP 

lOPT 

lOPTl 

I0PT2 

I0PT3 

I0PT4 

DISOPT 

GAMA 

LDI 


XNREP  =  NREP 
DSEEDl  =  DSEED 

C  INITIALIZE  THE  ARRAYS  USED  TO  COMPUTE  THE  MEAN  AND  STANDARD  DEVIATION 
C  OF  EACH  ESTIMATOR 

DO  30  J=1,NPHASE 
DO  30  1=1,4 

EST(I,J)  =  0. 0 
MLEWD(I,J)  =0.0 
MLESP(I,J)  =  0.  0 
REGEST(I,J)  =  0. 0 
PHREST(I,J)  =0.0 

30  CONTINUE 

C  COMPUTE  AND  STORE  THE  YJK  VALUES  UP  TO  1000 

YJK(l)  =  0.  0 
DO  40  1=1,999 

YJK(I  +  1)  =  YJK(I)  +  1.  O/I 
40  CONTINUE 

C  COMPUTE  AND  STORE  K  BAR  FOR  THE  EXPONENTIAL  REGRESSION  MODEL 

SUM  =0.0 
DO  50  I=1,NPHASE 
SUM  =  SUM  +  I 
REG(1,I)  =  SUM/ I 
50  CONTINUE 

C  MAJOR  REPETITION  OF  THE  SIMULATION  LOOP 
DO  500  REP=1,NREP 


✓  W-, 
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C  INITIALIZE  FAILURE  CAUSE  VECTOR  fSTCAUS)  AND  CCUMSF) 

C  COMPUTE  THE  INITIAL  SYSTEM  RELIABILITY 

REL  =  1. 

DU  60  I=1,NCAUSE 

NFCAL'S(I)  =  0 
REL  =  REL  *  A(I,1) 

DO  60  J=l,3 

CUMSF(J,I)  =  0 

60  CONTINUE 

C  INITIALIZE  COLUMN  (FAILURE  #  )  COUNTER  FOR  THE  WORKING  ARRAY  (A) 

J  =  1 

C  LOOP  TO  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 
C  AND  THE  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  EACH  PHASE 

DO  130  K=l,NrHASE 

SKIP  ACTUAL  COMPON'ENT  RELIABILITY  COMPUTATION  AFTER  FIRST  REP 
AND  FOR  FIRST  FAILURE 

IF(J. EO. 1)  GOTO  75 
IF(REP. GT. 1)  GOTO  75 
REL  =  1. 

C  IF  FIXED  RELIABILITY  OPTION  IS  SELECTED  THEN  PHASE  RELIABILITIES 
C  ARE  COMPUTED  AS  FOLLOWS 

IF  (FRELOP  .EQ.  1)  THEN 
DO  65  I=1,NCAUSE 
REL  =  REL-->A(I,J) 

NFCAUS(I)  =  0 
65  CONTINUE 

ELSE 

C  COMPUTE  NEW  ACTUAL  RELIABILITY  FOR  THE  COMPONENT  IN  PHASE  K 
DO  70  I=1,NCAUSE 

C  INCREASE  CAUSE  PR( SUCCESS)  IF  IT  CAUSED  FAILURE  IN  THE  PREVIOUS  PHASE 
C  COMPUTE  NEXT  PHASE  RELIABILITY  AND  REINITIALIZE  NFCAUS  (NOT  USED  IF 
C  FIXED  PHASE  RELIABILITY  OPTION  IS  SELECTED). 

IF(NFCAUS(I).  EQ.  1)  THEN 

A(I,J)  =  A(I,(J-1))  +  ((1.  -  A(I,(J-1)))*FRIMP) 
ELSEIF(NFCAUS(I).NE.  1)  THEN 
A(I,J)  =  A(I,(J-1)) 

ELSE 

ENDIF 

REL  =  REL*A(I,J) 

NFCAUS(I)  =  0 
70  CONTINUE 


61 


END  IF 


75  J1  =  1 

TRTOT  =  0.0 

C  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  A.ND  INCLUDI.NG  FAILURE  A.ND  THE 
C  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  THE  PHASE 

DO  120  L=1,NFAPH(  K) 

IF^REP.  GT.  1  :  GOTO  90 
IF(  Jl.  EG-  n  GOTO  35 
IF  ( FRELOP  .EG.  1)  GOTO  85 
DO  30  I=1,NCAUSE 
Ai  I.J)  =  Aa,(J-l)) 

80  CC.NTINUE 

85  A(  ISYSFR, J)  =  KEL 

AUFHASE.J)  =  K 

90  MIN  =  7,  2E75 

DO  no  I  =  1,NCAUSE 

C  A.SSIGN  TRIALS  FOR  CAUSES  WITH  PR(  SUCCESS)  =  0  OR  1 

IF(A(  I,J).GE.  1.  )  THEN 
A( ( I+NCAUSE) ,J)  =  7.2E75 
GOTO  100 

ELSEIFCAr  I  .,n.  EQ.  0.  )  THEN 
Ac ( I+NCAUSE) ,J)  =  1. 

GOTO  100 
ELSE 
END  IF 

C  CONVERT  UNIFORM  (0,1)  RANDOM  VARIABLE  TO  GEOME'FRIC  (#  TRIALS  UNTIL 
C  FAILURE  )  FOR  EACH  FAILURE  CAUSE.  RECORD  THE  MIN  TRIALS  FOR  THE 
C  CAUSES  AS  THE  SYSTEM  TRIALS  UP  TO  AND  INCLUDING  FAILURE  AND 
C  RECORD  THE  FAILURE  CAUSE 

A( (I+NCAUSE) ,J)  =  INT( 1. +(LOG(GGUBFS(DSEED))/LOG(A(I,J)))) 
100  IFfA(( I+NCAUSE) ,J). LE. MIN)  THEN 

MIN  =  A((I+.NCAUSE),J) 

IMIN  =  I 
ELSE 
END  IF 

110  CONTINUE 

A(IFAILC,J)  =  IMIN 
NFCAUS(IMIN)  =  1 

C  COMPUTE  THE  TOTAL  #  OF  TRIALS  FOR  THE  MLE  SINGLE  PHASE  ESTIMATE  AND 
C  INCREMENT  FAILURE  #  COUNTERS 

A(INTR,J)  =  MIN 

TRTOT  =  TRTOT  +  A(INTR,J) 

J  =  J  +  1 
Jl  =  Jl  +  1 
120  CONTINUE 


S'® 


**t^H^*  iLi 


c:  C'JMF'TE  TIiE  MLE  ESTLMATF.  QF  COMI’ONENT  REEIABILI'^Y  TV.IS  r!!ASE  AND 
N  NCMP'JTE  TEE  RINNING  SU:i  OF  ESTIMATES  AND  T!!E  SO:'  OF  ESTIMATFS  S'F'ARED 
C  FOR  COMPITATICN  OF  THE  MEAN  AND  STANDARD  DEVIATION  OF  THE  ESriMAFE 

PHRESTO.K)  =  CTRTOT  -  NFAFHi  K')  1 /TKTOT 
HLESPri.Kj  =  MLESP'l.K)  +  FHRESTC3,K) 

;iEESPr2,K)  =  ;iL£SF'2,K)  +  (  FHRESTC3,Kj**2) 

130  CONTINUE 

C  INTTIALIZE  THE  ADJUSTED  NUMBER  OF  FAILURES  TO  1  AND  THE  COUNT  OF  THE 
C  NUMBER  OF  TRIALS  SINCE  FAILURE  GR  DISCOUNTING  (NTRIALS(I)  )  TO  0 
C  I.N  PREPARATION  FOR  THE  DISCCUNTING  ROUTINE 

DO  140  J=1,NFAIL 

ACIADJF.J)  =  1. 

140  CONTINUE 

DO  150  I=1,NCAUSE 
NTRIALf!)  =  0 
130  CONTINUE 

C  DISCOUNTING  ROUTINE  REVIEWS  ALL  PAST  FAILURES  AND  CAUSES  TO  DATE 
C  AND  DETERMINES  IF  THE  DISCOUNTING  CONDITIONS  HAVE  BEEN  MET.  COMPUTES 
C  THE  ADJUSTED  FAILURES,  THE  ADJUSTED  OF  TRIALS  AND  YJK 

J  =  0 

DO  300  K=1,NPIIASE 
DO  200  L=i,NFAPH(K) 

J  =  J  +  1 

C  UPDATES  THE  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  EACH 
C  FAILURE  CAUSE 

ICAUSE  =  INT(A( IFAILC,J)+.  5) 

DO  160  r=l,NCAUSE 

IFdCAUSE.  EQ.  I)  THEN 
NTRIALCI)  =  0 
ELSEIFC ICAUSE. NE. I )  THEN 

NTRIAL(I)  =  NTRIALII)  +  INT( AC INTR,J)+,  5) 

ELSE 

ENDIF 

160  CONTINUE 
200  CONTINUE 

C  CHOOSE  DISCOUNTING  METHOD  TO  BE  USED 

IFCDISOPT. NE. 2)  GOTO  180 

C  PERFORM  LLOYD'S  FAILURE  DISCOUNTING  METHOD 

DO  170  1=1, J 

II  =  1NTCA(IFAILC,I)+.  5) 

IF(NTRIAL(I1).EQ.  0)  THEN 
A(IADJF,I)  =  1.0 


GOTO  170 


ELSE 

ENDIF 

C  THIS  IS  THE  MODIFIED  LLOYD  METHOD  OSING  A  DISCOUNT  INTERVAL.  THE 
C  ORIGINAL  DISCOUNT  .METHOD  MAY  BE  EMPLOYED  BY  SETTING  LDI  TO  ONE. 

ALD  =  I.NTr.NTRIAL(Il)/LDIj 
IF(ALD  .EQ.  0)  THEN 
AriADJF.I)  =  1.0 
GO  TO  170 
ELSE 

AriADJF.I)  =  1.  0  -  ((1. -GAMA)**a.  0/ALD)) 

ENDIF 

170  CONTINUE 
GOTO  210 

C  PERFORMS  STRAIGHT  PERCENT  FAILURE  DTSCOU.NTING  A.ND 
C  COMPUTES  THE  ADJUSTED  •■i  OF  FAILURES 


ISO  DO  IOC'  1=1.  J 

II  =  INT(  A(  IFAILC,I)+.  E’l 
IFINTRIALf  II).  EQ.  0)  THEN 
AdADJF.n  =  1. 

ELSE  IF (.  NTR I  AL(  1 1 ) .  GE.  N)  THEN 

A(IADJF,I)  =  ACIADJF.I)”' (1. -R)**(.NTRIAL(I1)/N)) 

ELSE 

ENDIF 

190  CONTINUE 

C  ADJUSTS  THE  #  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  THOSE  CAUSES 
C  THAT  HAVE  MET  OR  SURPASSED  THE  DISCOUNTi.NG  THRESHOLD 
C  FOR  THE  STRAIGHT  PERCENT  DISCOUNTING  METHOD 

DO  203  I=1,NCAUSE 

IF(.NTRIAL(  I).  GE.  N)  .NTRIAL(I)  =  MOD( NTRIAL(  I)  ,N) 

05  CONTINUE 

10  TADJT  =  0.0 
TYJK  =  0. 0 
TPYJK  =  0. 0 
K1  =  0 

DO  215  12=1,3 
DO  215  I=1,NCAUSE 
CUMSF( 12,1)  =  0 
215  CONTINUE 


C  COMPUTES  THE  ADJUSTED  ^4  OF  TRIALS  FROM  THE  ADJUSTED  4  OF  FAILURES 
C  AND  COMPUTES  THE  SUM  OF  THE  ADJUSTED  #  OF  TRIALS  FOR  ESTIMATE  COMP 

FREE  =  0.0 
LTRIAL  =  0 


C  IF  ADJUSTED  FAILURES  ARE  APPROACHING  0  THEN  ADJUSTED  TRIALS  MUST 
C  BE  PRE-SET. 


O 


DO  240  1=1, J 

IFf A( lADJF.I)  .LE.  .0000001)  THEN 
AdADJF.I)  =  .0000001 
E.NDIF 

AUADJT,!)  =  A(  I.NTR,I)/A(  lADJF.I) 

TADJT  =  TADJT  +  Afl.ADJT.I) 

CO.'IPUTE  YJK  FRO.M  THE  ADJUSTED  v‘  OF  TRIALS  AND  STORE  THE  SUM  F'.'R 
ESTIMATE  COMPUTATION,  USE  ARRAY  FOR  it  TRIALS  <  1000  AND  APPROX.  FOR 
VALUES  >  1000 

N1  =  NINT(A( lADJT,!)) 

IF(N1. LE. 1000)  THEN 

ACIYJK,!)  =  YJK(Nl) 

ELSEIF(  Nl.  GT.  1000)  THEN 
X=N1 
0=12^'--X 
T=X+1 
S=X+2 

A(  lYJK ,  I  )=(  EUL+(  LOG(  X) )+(  l/(  T'-'-'X) )  -(  l/(  Q*T) )  -(  l/CQ^T-^-'S ) ) ) 

ELSE 
END  IF 

C  DETERMINE  IF  A  PHASE  BOUNDARY  HAS  BEEN  REACHED  TO  BEGIN  ESTIMATE 
C  COMPUTATION 

IF(I. EQ. 1)  GOTO  223 

IF(A( IPHASE.I). NE. A( IPHASE ,( I-l)) )  THEN 

C  COMPUTE  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

MAX  =  0 
K1  =  K1  +  1 

C  DETERMINE  THE  FAILURE  CAUSE  WITH  THE  LARGEST  #  OF  FAILURES 

DO  220  I1=1,NCAUSE 

IF(CUMSF( 1 , II). GT. MAX)  THEN 
MAX  =  CUMSF(1,I1) 

ICOL  =  II 
ELSE 
END  IF 
220  CONTINUE 


C  COMPUTE  YJK  VALUE  FOR  THE  CURRENT  PHASE  ESTIMATE 

IFCCUMSFI 1, ICOL). LE. 1000)  THEN  >J 

AHATL  =  YJK((:UMSF(1,1C0L))  V 

ELSEIF(CUMSF( 1, ICOL).  GT.  1000)  THEN  ^ 

X  =  CUMSF( l.ICOL)  -O 

Q=12--X  Iv 

» 


T=>M-1 

S=X-t-2 

AHATL=(,  EL'L+(  LOG(  X) )+( l/(  2-\X) ) -<  l/(.  Q*T) )  -(  1/  ( Q*T*S  l  j 

ELSE 

ENDiF 

IX  =  Cl'MSFf  1  ,  ICOL)  +  CUMSFO.ICOL) 

IF( IX. LE.  1000)  THEN 

AHATU  =  YJKdXj 
ELSEIFdX.  GT.  lOCO)  THEN 
X  =  IX 
0=12'>X 
T=X+1 
S=Xi-2 

AHATU=(.EUL+(  L0G(  X) )+( l/(  2*X) )  -(  1/(Q*T) )  -(  l/(  Q*r^S) ) ) 

ELSE 

ENDIF 

C  COMPETE  CERRENT  PHASE  RELIABILITY  ESTIMATE 

AHAT  =  AHATU  -  AlIATL 
OREL  =  1.  0  -  EXP( -AHAT) 

X  =  CUMSFd.ICOL)  +  CUMSFO.ICOL) 

C  COMPUTE  AND  STORE  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

PREL  =  ((LTRIAL*PREL)/X)  +  ( ( ( X-LTRIAL)*CREL) /X) 

LTRIAL  =  CUMSFC l.ICOL)  +  CUMSF( 3 , ICOL) 

C  COMPUTE  THE  PHASE  AND  GLOBAL  AVERAGE  FOR  YJK  USED  IN  THE  EXPONENTIAL 
C  REGRESSION  ESTIMATES  ARE 

REG(2,K1)  =  TYJK/( I-l) 

R£G(3,K1)  =  TPYJK/NFAPH(K1) 

TPYJK  =  0.  0 

ENDIF 

C  COMPUTE  THE  NUMBER  OF  FAILURES  AND  SUCCESSES  FOR  EACH  FAILURE  CAUSE 
C  USED  IN  T!1E  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

225  ICAUSE  =  INTf A( IFAILC,I)+.  5) 

DO  230  I1=1,NCAUSE 

CUMSF(2,I1)  =  CUMSFC 2, II)  +  INT( AC INTR, I )  +  .5) 

CUMSFC 3, 11)  =  CUMSFC 3, II)  +  N1 
230  CONTINUE 

CUMSFC  1 , ICAUSE)  =  CUMSFC 1 , ICAUSE)  +  1 
CUMSFC 2, ICAUSE)  =  CUMSFC 2 , ICAUSE )  -  1 
CUMSFC 3, ICAUSE)  =  CUMSFC 3 , ICAUSE )  -  1 
TPYJK  =  TPYJK  +  AC  [YJK, I) 

TYJK  =  TYJK  +  Ad  YJK,  I) 

240  CONTINUE 
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RET-LAT  CCMPUTATICN’S 


THL  WOODS  WEIGHTED  A',  EEAC^ 


L.'' 


■'mA' 


FINAL  PNASE 


MAX  =  0 

El  =  K1  +  1 

DO  2X3  H=l,NGAl'SE 

if:  Cl':iSF(  l  ,  1 1  ; .  GT.  max)  then 

MAX  =  OMMSF: 1,11) 

ICOL  =  II 
ELSE 
END  IF 
245  C0NTi;-;iE 

IF(  CMMSFC  1 ,  ICOL).  LE.  1000')  THEN 
AHATL  =  YJKl  CCMSFf I , ICOL) ) 

ELSEIFt  CUMSFf 1 , ICOL). GT.  1000)  THEN 
X  =  CUMSFC 1 ,ICOL) 

Q=12*X 

T=X+-1 

3=X+2 

Ar.ATL=i  E’JL-f-(  LOG:  X)  1,  (.  L'^'X  j  t-i  1  ■  '  'J-’r'  1  (.0'’''T''S  j  ) ) 


ELSE 
END  IF 

IX  =  CMMSFC  1,  ICOL)  +  CL'MSF(3,IC0L) 

IF( IX. LE. 1000)  THEN 
AHATU  =  YJK(IX) 

ELSEIFC  tX. GT.  1000)  THEN 
X  =  IX 
Q=12*X 
T=X+1 
S=X+2 

AH ATU= ( EU L+  (  LOG  ( X ) ) + (  1  /  ( 2*X ) )  -  (  1  /  ( Q*T ) )  -  (  1  /  ( S ) ) ) 

ELSE 

ENDIF 


AHAT  =  AHATU  -  AHATL 
OREL  =  1. 0  -  EXF( -AHAT) 

X  =  CUMSFC 1, ICOL)  +  CUMSFC 3, ICOL) 


PREL  =  CCLTRIAL*PREL)/X)  +  C ( CX-LTRIAL)*CREL) /X) 

LTRIAL  =  CUMSFC 1, ICOL)  +  CUMSFC 3 , ICOL) 

REGC2,K1)  =  TYJK/CJ) 

REGC3,K1)  =  TPYJK/NFAPHCKl) 

PHRESTC1,K)  =  PREL 

C  COMPUTE  THE  MLE  ESTIMATE  OF  PHASE  RELIABILITY  USING  DISCOUNTING 
PHRESTC2,K)  =  CTADJT  -  J)/TADJT 


C  COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE  BEGINNING  WITH  B  HAT 


S  -L  *-  N.  •.  •.  V 


•.  N.  V 


SCM  =  0.0 
SUMS  =  Q. 0 

IF  (rl.  £Q.  1)  GOTO  252 
DO  250  I  =  1,K 

SUM  =  SUM  +  ai-REGt  l,Kn'''RE0r3,n  ' 

SI:MS  =  SUMS  +  ((I-R£G(  1,K);**2, 

250  CQNTl.NUE 

REGf^.K)  =  SUM/ SUMS 

C  COMPUTE  A  HAT 

REG(.5,:<)  =  REG(2,K)  -  ( REGC  u ,  K  j*P.EG(  1 ,  K) ) 

C  COMPUTE  AND  STORE  THE  EXPONENTIAL  REGRESSION  ESTLMATE 

PUREST,  -sK)  =  1.0  -  EXP( -(REG(5,K)  +  I REG(  4 , K) ••'K) ) ) 
If(l':!REST(4,K).  LT.  0.  0)  PHRESTC A,K)=0.  0 
GOTO  25: 

252  FHRESTf =  i.  o  -  EXFf -REGf  2  . 1  ; ) 

IFlPHRESTf  ^,K).  ET.  0.  0)  P!1KE3T(>  , H ,=0.  0 

C  STORE  THE  RU.NNTN'G  SUM  OF  THE  ESTir!ATES  FOR  THE  CURRENT  PHASE  AND  THE 
C  RUNNING  SUM  OF  THE  ESTLMATES  SQUARED  FOR  COMPUTATION  OF  THE  MEAN  AND 
C  STANDARD  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  RELIABILITY  GRCVTH 
C  MODEL 

255  EST(1,K)  =  EST(1,K)  +  PilRESTfl.K) 

EST(2,K)  =  EST(2,K)  +  ( PHREST( 1 ,K)**2) 

MLEWD(1,K)  =  MLEWD(1,K)  +  PHRF.ST(2,K) 

MLEWD(2,K)  =  MLEWD(2,K)  +  ( PHREST( 2 ,K)**2) 

REGESTd.K)  =  REGEST(i,K)  +  PHREST(4,K) 

REGEST(2,K)  =  REGEST(2,K)  +  ( PHREST( 4 ,K)**2) 

C  STORE  THE  ACITJAL  PHASE  RELIABILITY 

AREL(K)  =  A(ISYSPR,J) 

C  PRINT  LNTERMEDIATE  OUTPUT  IF  REQUESTED  AND  THE  NUMBER  OF  REPETITIONS 
C  IS  NOT  GREATER  THAN  5 

IF( lOPT. NE. 1)  GOTO  300 
IF(REP.  GT.  5)  GOTO  300 

WRITE! 20, 1000)  REP,K 

1000  FGR.MAT(T16,' REPETITION  NUMBER:  ',14,'  PHASE  NUMBER;  ',14) 
WRITE(20,1010)  A(ISYSPR,J) 

1010  FORMAT! 22X, 'ACTUAL  COMPONENT  RELIABILITY;  ',F7.5) 

WRITE! 20, 1020)  PHREST(1,K) 

1020  FORMAT! 20X, 'PREDICTED  COMPONENT  RELIABILITY:  ',F7.5) 
WTITE!20,1022)  FHREST(2,K) 

1022  FORMAT! 20X, 'MLE  ESTIMATE  USING  DISCOUNTING:  ',F7.5) 

WRITE! 20, 1025;  PUREST! 3, K) 

1025  FORMAT! 18X, 'MLE  ESTIMATE  OF  PHASE  RELIABILITY:  ' ,F7. 5) 

WRITE!20, 1027)  PHREST!4,K) 


y.j 


1027  FORMAT! 14X, ' REGRESSION  ESTIMATE  OF  PHASE  RELIABILITY:  ',F7.5) 
WRITE'' 20, 1020) 

1030  F’JRMATf  '  '  .  '  '  ) 

DO  260  I=1,NCAUSE 

WRITE!  20  ,  loss')  I ,  AC  I ,  J)  ,A(  ( I+NCAL'SE')  ,  J  ) 

1.035  FORMAT!  12X, 'CAUSE:  ',13,'  PR(  SUCCESS  ) :  ’.F7.6,’  #  TRIALS:  '. 

CFIO. 0) 

260  CQNTI.NUE 

WRITE! 20, 1036) 

1036  FORMAT!'  ') 

WRITE:' 20,  lOAO) 

lOAO  F0RMAT!4X,  '  FAIL  j>',3X,'FAIL  CAUSE ',  3X, ' TRIALS ',  3X ,' AUJ  FAIL', 3 
CX,'ADJ  TRIALS' ,7X, 'YJK' ) 

DO  270  1=1, J 

WR ITE! 20 , 1050 ) I , A! IFAILC , I ) , A! INTR , I ) , A! lAUJF , I ) , A! lADJT , 11 , Ai  lYJK 
C,!) 

105  0  FCRMAT!4X, I3,8X,F3. 0 , 7X , F8.  0 , 4X ,F3.  6 , 4X , F12.  0 , 3X ,F 1 1.  4 ) 

270  CONTINUE 

WRITE! 20. 1060) 

1160  FOR.MAT!  '  '  ,/.  /) 

100  ''ONTINl’E 

C  FRINT  EACH  OF  THE  3  ESTIMATES  TO  THEIR  AfPROPRLATE  OUTPUT  FILE 
C  IF  REQUESTED 

IF! lOPTl.  NE.  1)  GOTO  401 
AGO  WRITE!40,2000)  ! PHREST! 1 , I ) ,  I=1,NPHASE) 

401  IF! I0PT2.  NE.  1)  goto  402 

WRITE! 5 0,2000)  ( FHRESTi 2 , I ) ,  I=1,NPHASE) 

402  IF! I0PT3. NE. 1)  GOTO  403 
WRITE!60,2000)  ! PUREST! 3 , I ) ,  I=1,NFHASE) 

h03  IF! I0PT4. NE. 1)  GOTO  500 

WRITE! 70,2000)  ! PHREST! 4 , I ) ,  I=1,NFHASE) 

2000  FOR.MAT!  '  '  ,30!F7.  6:  IX)) 

500  CONTINUE 

C  UPON  COMPLETION  OF  ALL  REPETITIONS,  COMPUTE  THE  MEAN  AND  STANDARD 
C  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  PHASE  SKIPPING  COMPUTATIONS  IF 
C  ONLY  ONE  REPETITION  IS  REQUIRED 

IF  !NREP. L£.  1)  GOTO  601 

DO  600  I=1,NPHASE 
EST!3,I)  =  EST! 1 ,I)/XNREP 
MLEWD!3,I)  =  MLEWD! 1,I)/XNREP 
MLESP!3,I)  =  MLESP! 1,I)/XNREP 
REGEST!3,I)  =  REGEST! 1,I)/aNREP 

EST! 4 , 1 )  =  SORT! ! EST! 2,1)-! XNREP*! EST( 3 , 1 )**2 ) ) ) / ! XNREP - 1 1 ) 

MLEWD!  4,1)  =  SQRT! ! MLEWD!  2,1)  -!  XNREP'i!  MLEWD!  3 , 1  )**2) ) )  /  !  XNREP  - 1 ) ) 
MLESFC  4,1)  =  SORT! ! MLESP! 2 , 1 )-! XNREP*! MLESP! 3 , 1 ) **2 ) ) ) / ! XNREP- 1 ) ) 
HUGEST! 4 , I )=SQRT! ! REGEST! 2 , I )-! XNREP*! REGEST! 3 , I )**2 ) ) ) / ! XNREP- 1 ) ) 
600  CONTINUE 


C  PRINT  THE  FINAL  OUTPUT  TABLE  TO  A  FILE 


69 


5: 

601 

WRITE! 

30. 

3000) 

>■ 

3000 

FORMAT! ' 0 

'  ,T47 

WRITE! 

30, 

3010) 

3010 

FORMAT 

!'- 

'  ,T54 

WRITE! 

30, 

3020) 

3020  FORMAK 'o' ,T47, 
IF  I  FRELOP  . EQ. 


'DISCRETE  RELIABILITY  GROWTH  SIMULATION' 

' MODEL  PARAMETER  SUMMARY ' ) 

MCAUSE 

'NUMBER  OF  POSSIBLE  FAILURE  CAUSES  '.14) 
1)  GOTO  4000 


) 


WRITE(  30,3030) 

3030  FORMATC 'O’ ,T38, ’CAUSE  NUMBER’ ,T64, ’SINGLE  TRIAL  PR(  SUCCESS  )  FOR 
CPHASE  1') 

DO  3050  ;i=l,NCAUSE 
WRITE(30,3040)  M,A(M,1) 

3040  FORMAT!'  ' ,T43 , 12 ,T79 , F8. 6) 

3050  CONTINUE 

WRITE! 30, 3060)  FRIMP 

3060  FORMAT! 'O' ,T37, 'FRACTION  CAUSE  RELIABILITY  IMPROVES  AFTER  FAILURE 
C  ,F8. 6) 

5000  WRITE! 30, 3080)  NPHASE 

3030  FORMAT! '-' ,T48, 'NUMBER  OF  PHASES  IN  THE  SIMULATION  ’,12) 

WRITE! 30. 3090) 

3090  FOR.MATf  '  O'  ,T42,  '  PMSE  NUMBER' ,T39 ,’ NUMBER  OF  FAILURES  IN  THE  FIRST 
C  PHASE' ) 

DO  3110  M=l, NPHASE 
WRITE!30,3100)  M,NFAPH!M) 

3100  FORMAT!'  ' ,T43 , 12 ,T73 , 12) 

3110  CONTINUE 

WRITE! 30,3120)  NFAIL 

3120  FORMAT! 'O' ,T51, 'TOTAL  NUMBER  OF  FAILURES  ’,14) 

IF!DISOPT. EQ. 2)  GO  TO  3160 
WRITE!30,3130) 

3130  FORMAT! '-’ ,T38, 'DISCOUNTING  PERFORMED  USING  THE  CONSTANT  FRACTION 
CMETHOD ' ) 

WRITE! 30, 3 140)  R 

3140  FORMAT! 'O' ,T44, 'FRACTION  EACH  FAILURE  IS  DISCOUNTED  ',F8.6) 

WRITE! 30, 3150)  N 

3150  FORMAT!'  ' ,T33 ,' NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  DISCOUNT 
C  IS  APPLIED  ' ,14) 

GO  TO  3190 
3160  WRITE!30,3170) 

3170  FORMAT! '-' ,T44, 'DISCOUNTING  PERFORMED  USING  THE  LLOYD  METHOD') 
WRITE! 30,3180)  GAMA 

3130  FORMAT! 'o' ,T39, 'PERCENT  C.  I.  (  USED  AS  DISCOUNT  FRACTION  )  ' ,F8.  6 
C) 

WRITE! 30, 3 135)  LDI 

3185  FORMAT! 'O' ,T50, 'LLOYD  DISCOUNT  INTERVAL:  ’,13) 

3190  WRITE!30,3200)  DSEEDl 

3200  FORMAT! '-’ ,T46, 'RANDOM  NUMBER  SEED  USED  ' ,F15. 2) 

WRITE! 30, 3210)  NREP 

3210  FORMAT! 'O' ,T37, 'NUMBER  OF  REPETITIONS  OF  THE  SIMULATION  PERFORMED 
C  ,17) 

WRITE! 30, 3220) 

3220  FORMAT! ' 1 ' ,T61 , ’ ESTIMATOR: ’ ) 

WRITE! 30, 3230) 

3230  FORMAT! 'O' ,T48, 'SINGLE  PHASE  MLE  WITHOUT  DISCOUNTING') 

WRITE! 30, 3 240) 

3240  FORMAT! ' -'  ,T60, 'MEAN' ,T83, 'ESTIMATE' ,T109, '95  %'  ) 


V 


WRITEC 30,3250) 

3250  FCRMAK '  ',T12,' PHASE  NUMBER' ,T29 ACTUAL  RELIABILITY' ,T52 FFEDIC 
CTED  RELIABILITY' ,T7 8, ' STANDARD  DEVIATION'  ,T10 1 CONFIDENCE  INTERVA 
CL'  ) 

C  COMPUTE  C.  I.  FOR  SINGLE  PHASE  MLE 

DO  3270  M=1,NPHASE 

Cl  =  ( 1. 96*MLESP(4,M))/SQRT(XNREP) 

CIU  =  MLESPf3,M)  +  Cl 
CIL  =  MLESP(3,M)  -  Cl 

WRITE(30,3260)  M,AREL(M) ,MLESPf 3 .MI ,MLESP(4,M) , CIL, CIU 
3260  FORMATC 'o' ,T17,I2,T34,F8. 6,T58,F8. 6,T82,F9. 6,T99 , ' (  '.F8.6,'  ,  ',F 
C3.6,'  )') 

3270  CONTINUE 

WRITE(30,3220) 

WRITE! 30, 3280) 

3280  FORMATC 'O' ,T42, 'MAX  LIKELIHOOD  ESTIMATE  USING  DISCOUNTED  FAILURES' 

r  y 

■WRITE!  30, 3240) 

'WRITE!  30, 3250) 

C  COMPUTE  C.  I.  FOR  MLE  WITH  DISCOUNTING 

DO  3290  M=1,NPHASE 

Cl  =  (1. 96*MLEWD(4,M))/SQRT(XNREP) 

CIU  =  MLEWD(3,M)  +  Cl 
CIL  =  MLEWD(3,M)  -  Cl 

WRITE(30,3260)  M , AREL(M) ,MLEWD( 3 ,M) ,MLEWD( 4 ,M) , CIL, CIU 
3290  CONTINUE 

WRITEC 30, 3220) 

WRITEC 30, 3300) 

3300  FORMATC 'O' ,T38, 'WEIGHTED  AVERAGE  ESTIMATE  USING  FAILURE  DISCOUNTIN 
CG'  ) 

WRITEC 30, 3240) 

WRITEC30,3250) 

C  COMPUTE  C. I.  FOR  WOODS  WEIGHTED  AVERAGE  ESTIMATES 

DO  3310  M=1,NPHASE 

Cl  =  C 1. 96*ESTC4,M))/SQRTCXNREP) 

CIU  =  ESTC3,M)  +  Cl 
CIL  =  ESTC3,M)  -  Cl 

WRITEC30,3260)  M,ARELCM) ,ESTC3,M) ,ESTC4,M) , CIL, CIU 
3310  CONTINUE 

WRITE! 30, 3220) 

WRITEC30,3320) 

3320  FORMATC 'O' ,T43, 'REGRESSION  ESTIMATE  USING  DISCOUNTED  FAILURES') 
WRITE! 30, 3240) 

WRITE! 30,3250) 

C  COMPUTE  C.  I.  FOR  EXPONENTIAL  REGRESSION  ESTIMATES 

DO  3330  M=1,NPHASE 

Cl  =  C 1. 96*REGESTC4,M))/SQRTCXNREP) 


iSiifily 


V  ^  *  4 


CIU  =  REGEST(3,M)  +  Cl 
CIL  =  REGESTO.M'i  -  Cl 

WRITE(30,3260)  M,AREL(M) ,REGEST( 3 ,M) ,REGEST(4,M) , CIL, CIU 
3330  CONTINUE 

WRITE! 30,3340) 

3340  FORMAT!' l' ,T59 ,' RECAPITULATION* // ) 

WRITE! 30,3350) 

3350  FORMAT! ' ,T3, 'PHASE' , Til ACTUAL' ,T28, 'MEAN' ,T38. 'EST' ,T53, 'MEAN' 
C,T63, 'EST' ,T78, 'MEAN' ,T88, 'EST' ,T103, 'MEAN' ,T113, 'EST' ) 

WRITE! 30, 3360) 

3360  FORMAT! '  ' ,T11  'RELIAB' ,T28, 'WGT' ,T3S, 'STD' ,T53, 'MLE' ,T63, 'STD' ,T7 
C7, 'PHASE' ,T88,'STD' ,T103,’REG' ,Til3,’STD') 

WRITE! 30, 3370) 

3370  FORMAT!'  ' ,T28,'AVG' ,T35 ,' DEVIATION' ,T53,'W/D' ,T60 ,' DEVIATION' ,T78 
C,'MLE' ,T85, 'DEVIATION' ,T103,'EST' ,T1 10 ,' DEVIATION ' ) 

WRITE! 30,3375) 

3375  FORMAT!'  ' ,T28, 'EST' /) 

DO  650  I=1,NPHASE 

WRITE!30,3380)T,AREL! I) ,EST!3,I) ,EST!4,I) ,MLEWD!3,I) ,MLEWD!4,I) , 
CMLESP!3,I) ,MLE3P!4,I) .REGEST!3, H ,REGEST!4.I) 

3380  FORMAT! 'O'  ,T4 , 13 ,T1 1 ,F7. 6 ,T26 .F7. 6,T36,F7.  6,T51,F7.  6,T6i,F7.  6.T76, 
CF7. 6,T36,F7.  6,T101,F7.  6,T111,F7.  6) 

650  CONTINUE 
GO  TO  6000 

4000  WRITE! 30, 4010) 

4010  FORMAT! IX,//, T50, 'FIXED  PHASE  RELIABILITY  OPTION') 

WRITE!30,4020) 

4020  FORMAT! '-' ,T38, ' PHASE  NUMBER' ,T78 ,' ACTUAL  RELIABILITY') 

DO  4030  M=1,NPHASE 

WRITE!30,4040)  M,AREL!M) 

4040  FORMAT! 'O' ,T41 , 12 ,T83 ,F8.  6) 

4030  CONTINUE 
GO  TO  5000 
6000  CONTINUE 
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APPENDIX  C  GRAPHICAL  RESULTS 


This  appendix  contains  the  results  of  many  of  the  various  runs  done  with  the  sim¬ 
ulation.  These  results  appear  in  tabular  format  in  the  output  file  but  have  been  reduced 
to  graphical  form  for  ease  of  understanding.  Below  each  graph  of  the  estimates  produced 
by  the  growth  models  are  graphs  of  the  standard  deviations  of  each  estimate  by  phase. 
This  appendix  is  ordered  by  reliability  growth  pattern  with  Pattern  I  first  and  the  re¬ 
mainder  following  sequentially. 
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Figure  40.  Pattern  II,  F  “  .50, 1-3 
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Figure  48.  Pattern  III,  F 
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Figure  49.  Pattern  III,  F  ■  .25, 1  »  6 
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Figure  52.  Pattern  III,  F  =  .75, 1  =  3 


Figure  53.  Pattern  III,  F  =  .75, 1  =  6 
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Figure  65.  Pattern  V,  F  =  .25,  I  =  I 
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Figure  68.  Pattern  V,  F  =  .50,  I  =  3 
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Figure  85.  Pattern  VII,  F  =  .25,  I  =  1 
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Figure  98.  Pattern  VIII,  F  =  .50,  1  =  3 
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